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MULTI-MEDIA INFORMATION RECORD 
DEVICE, AND A MULTI-MEDIA 
INFORMATION PLAYBACK DEVICE 

BACKGROUND OF THE INVENTION 
(1) Field of the Invention 

The present invention relates to a multi- media informa- 
tion record device and a multi-media information playback 
device: the multi-media information record device records 
multi-media information like dynamic /still picture, 
graphics, sound and the like in a storage device and a 
recording medium provided to a data server; the multi-media 
information playback device plays back the multi-media 
information inputted via a recording medium and a trans- 
mission line. 15 

{2} Description of the Related Art 

Recently. TV game sets, electronic publications, quality 
karaoke sets and video on demand devices are used as 
read-only multi-media information playback devices. These 
multimedia information playback devices play back multi- 2 o 
media information recorded in a optical disc (CD-ROM) and 
a digital video disc (DVD>. and also multi-media informa- 
tion transmitted from a data server via a network and the 
like. 

The following explanation is about "Denshi Kamishibai- 2 s 
Electronic Picture-Card Show" disclosed in Japanese Laid- 
open Patent Application No. 5-2864S. which is an example 
of the conventional multi-media information playback 
device mentioned above. FIG. 1 is an example of a data 
configuration of multi-media information of a conventional 30 
multi -media information playback device, by which images 
and sound are synchronously played back. 

Data and information shown in a data area section and 
control information secdon of HG. 1 are recorded in the 
recording medium like a digital disc. Control informadon 35 
shown in the control information section is recorded in each 
block composed of predetermined amount of data shown in 
the data area section, in which sound data and still picture 
data mingle. In the data area section of FIG. 1, An (n is a 
natural number) represents sound data: VnB and VnC rep- 40 
resent still pictures A and B. respectively; GLUT (color look 
up table) represents GLUT data: NOPn represents data to 
which no operation is applied. In the control informadon 
section. (1) represents a data write command: {2; represents 
a GLUT data command; (3) represents a data display com- 45 
raand; (4) represents an audio data cornmand; (5) represents 
a command of no operation. In the screen image section. A 
and B represent still pictures A and B. respectively. Both of 
them are displayed on the screen. 

Each command in the control information section is 50 
executed in order from the left side to the right side. First 
audio data Al, which is digitaiized sound information, is 
transferred to audio data memory by the control information ■ 
(4). and outputted as an analog audio signal via an audio D/A ! 
(digital/analog j converter. Video data V1B is written in a 55 i 
predetermined position of video memory of V42B from 
VLB. after the write area of a still picture is designated by 
the control information (1). The GLUT data updates the 
screen image by the control information (2). Based on a 
GLUT memory, contents of the video memory are converted 60 
to an analog video signal by the video D/A converter, so that 
the contents are displayed on the screen. While the screen 
image and an audio signal A44 in FIG. 1 are synchronized 
a still picture A can be switched to a still picture B. Although 
the succeeding still pictures G. D. E . . . are not shown tn 65 
FIG. 1, the same processing is carried out to play back i 
display screens and sound synchronously. 
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The next explanation is about a general playback process- 
ing of a multi-media information playback device. Multi- 
media information playback device places a line end signal 
and a page end signal, which are already inserted in the 

5 digitalized image signals, at the end of a scanning line 
(normally more than 500) and a page (contents of a screen 
at every Vx second) in the display device like Cathode Ray 
Tube and Liquid Crystal Display, respectively, and displays 
the scanning lines on the proper pages, so that image signals 

10 recorded in the recording medium like a CD-ROM and a 
DVD are displayed at predetermined positions on the screen 
of the display device. Multi-media information playback 
device quadruples a size of an image to be displayed on the 
screen by using two pixels vertically and horizontally, which 

1 5 forms the smallest square of four pixels, for each pixel in the 
display device, or by interpolation. This device also reduces 
an image to a quarter size, by skipping a pixel vertically and 
horizontally. 

Also the multi-media information playback device rotates 
20 an image to be displayed on the screen by converting 
coordinates of each pixel using trigonometric function. As 
pixels are more dense in the recording medium than in the 
display device, predetermined adjustment is applied to the 
recording medium so that a proper image is displayed based 
25 on the pixel density of the display device, in time of normal 
display. So practically, there's no problem when an image 
which is stored in the CD-ROM as a small still picture is 
enlarged as much as possible on the screen of the display- 
unit. 

30 Generally, the multi-media information playback device 
is equipped with necessary programs to play back the 
multi-media information stored in the recording medium. 
Sometimes, a different program is already written in the 
multi-media information so that commands which are added 

35 to the multi-media information or which are changed are 
executed according to the program, In this case, signs which 
are similar to T** and are given to the different program 
which is described by the multi-media information, so that 
the multimedia information playback device recognizes the 

40 program to be executed. 

Moreover, the multi-media information playback device 
displays a still picture, which does not change at all during 
the display time, in the display device by repeating the steps 
of reading and displaying the display data stored in a VRAM 

45 (video RAM) every Vy> second or so. On the other hand, 
dynamic pictures are displayed by repeating the steps of 
updating the display data stored in the VRAM, reading and 
displaying the updated display data every Vx> second or so. 
Therefore, one storage unit of page data which shows each 

50 screen image of the display device is about Vx> second for 
dynamic pictures. 

Sound which is played back synchronously with the 
dynamic pictures can be delimited by V20 second or so and 

55 stored with each page data which consists the dynamic 
picture. Therefore, the multi-media information playback 
device is equipped with a storage unit which stores sound 
data of at least l /» second so that the sound is successively 
played back along with the dynamic pictures. 

50 As for DVD, details can be found in the next reference; 
"Movies can be stored in a hard disc photo-electric mag- 
netic and phase change", featured in Jul. 18. 1994 issue of 
"Nikkei Electronics No, 613". pages 76-Si. 

However, in the conventional art mentioned above, data 
65 about movies, sound, and their controlling, are stored fix- 
edly. Specifically, in the case of image data, data for the 
whole screen image including the background is stored as 
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one still picture data. Therefore, when pan of the screen 
image is changed by. e.g. moving a part of the still picture 
to a different position on the screen, enlarging or reducing a 
part of the still picture, more still picture data coping with 
the change has to be stored in the CD-ROM or the DVD. So 5 
the maximum number of the still pictures which can be 
displayed is limited by the storage capacity of the CD-ROM 
and the DVD. Moreover, each time the still picture data is 
displayed, they have to be read from the CD-ROM and the 
DVD. so the number of still pictures which can be displayed ic 
per unit time is limited by the data transfer rate between the 
CD-ROM or the DVD and a memory. 

When the audio data, the still picture data, the dynamic 
picture data and the control information are stored in dif- 
ferent storage areas, the multi-media informadon playback 15 
device has to access different storage areas of the CD-ROM 
or the DVD in order to read the control information and each 
data used for it every time it executes the control informa- 
tion. This results in bad read efficiency. Also, load of the 
multi-media information playback device for controlling the 20 
internal memory is large since data are separately read into 
the memory provided to the multi-media informadon play- 
back device from the different storage areas of the CD-ROM 
and the DVD. 

25 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a 
multi -media information playback device, in which readout 
of display data from a recording medium is not necessary ^ 
each rime a pan of a screen image is changed and displayed 
the display data can be created easily, and the load on a CPU 
controlling a memory is small. 

The above object may be fulfilled by the multi-media 
information playback device which shows the following 35 
features. 

(1) A multi-media information playback device which 
plays back multi-media information recorded in a recording 
medium comprises a disc having page information succes- 
sively stored in a storage area, the page information includ- 40 
ing a partial image list having a pair of partial image 
information showing a partial image of a screen image and 
display state information showing display state of the partial 
image, and the page information also including control 
information which describes a command to be executed « 
according to an input signal, the control informadon includ- 
ing a display state change command instructing to change 
the display state of the partial image: a page information 
read unit for reading the page information from the disc; a 
display data creation unit for creating display data of the 50 
screen image by placing the partial image shown by the 
partial image informadon on a screen; a display unit for 
displaying each display data created by the display data 
creation unit: an input unit for receiving an input signal from 
an operator, and a command execution unit for detecting the 55 
input signal inputted by the operator, judging whether the 
input signal corresponds to the command described by the 
control information in the page informadon read by the page 
information read unit, and instructing the display data cre- 
ation unit to create new display data by changing the display $c 
state of the corresponding partial image according to the 
display state change command when the input signal corre- 
sponds to the display state change command: wherein the 
display data creadoa unit creates new display data according 
to the coinrnand execution unit. 65 ! 

According to the above construction, image information [ 
and display state of partial images constituting one screen 
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Image are shown by the page information successively 
recorded in the storage area on the disc. So in time of 
changing and displaying a part of a screen image, what is 
needed is only creating display data by changing the display 
5 state of the corresponding partial image according to the 
control information. The display data can be changed easily 
in this way. 

Also, a part of the screen image of display data can be 
changed only by the page information, so that the display 

1C data does not have to be read from the recording medium. So 
there is no need to record display data, whose part of the 
screen image is changed, in the recording medium before- 
hand. Therefore, the number of display data which can be 
displayed per unit time is not limited by the data transfer rate 

l5 between the disc and the memory. 

Moreover, information needed for changing and display- 
ing only a part of the screen image using partial images is 
shown by the partial image list in one page information. So 
the load on a CPU which controls memory in the multi- 

20 media information playback device is small compared to the 
case in which necessary information is loaded to the 
memory of the multi -media information playback device. 
When a new image screen composed of different partial 
images in the multi-media information playback device is 

25 displayed, new page information which describes the partial 
images is read, and display data for showing that image 
screen is created from the page information. Therefore, new 
display data can be created by only one head seek which 
happens in time of reading new page information. Compared 

50 to the case in which partial image informatioa is read from 
several places on the disc (which unit head seek happens 
several times), the readout of the partial image rnformation 
can be carried out in short time. That means new image 
screen is displayed in shorter time than before. 

35 (2) The page information may include the partial image 
list, the control information, and the time sequence control 
information including a name of a second time sequence 
information as time sequence information to be played back 
according to the time sequence processing start command, 

40 pairs of IDs of sound information and image information to 
be played back at predetermined times, and the playback 
state information which describes at least one of dynamic 
picture and animation should be displayed in an area relating 
to the partial image shown by the partial image information; 

45 wherein the time sequence information may include the 
second time sequence information comprising sound infor- 
mation divided into equal predetermined sizes and image 
information showing an image of at least one of the dynamic 
picture and the animation, the sound information and the 

50 image information being placed in order of playback corre- 
sponding to the elapsed time; wherein the time sequence 
processing execution unit may include a second time 
sequence processing execution unit for designating sound 
information and image information in the second time 

55 sequence information corresponding to the elapsed time at 
predetermined times when the second time sequence infor- 
mation is shown by the time sequence processing control 
information, and instructing the display data creation unit to 
create new display data by placing the partial image read by 

6o the time sequence information read unit in an area shown by 
the playback state information: wherein the display data 
creation unit may create new display data according to the 
second time sequence processing execution unit 
According to above construction, the time sequence infor- 

65 mation including the second time sequence information, 
which describes sound information and image information 
in order of playback corresponding to the elapsed tune, is 
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stored successively in the storage area on the disc. So head 
seek does not happen when the sound information and image 
information are successively read corresponding to the 
elapsed time. Therefore, the multi-media information play- 
back device can read the sound information and the image 5 
informauon at high speed, so breaks does not happen, in 
sound, animation and/or dynamic pictures, while the multi- 
media information is played back. 

(3 J A mulct-media information record device for recording 
multi -media information in a disc may comprise a partial ^ 
image storage unit for storing the partial image information 
of at least one partial image in one partial image file, the 
partial image information being a part of the screen image; 
a partial image list input unit for receiving input of a name 
of at least one partial image file in the partial image storage l5 
unit and display state information showing display state of 
the partial image shown by the partial image information in 
the partial image file; a page information storage unit for 
having a plurality of separate small storage areas, each of 
which is used for storing at least a piece of page information: 2 c 
a partial image list write unit for writing a pair of the partial 
image information and the display state information in a first 
small storage area in the page information storage unit along 
with the IDs of the partial image information and the display 
state information, after reading partial image information 35 
from the partial image file whose name is inputted, the 
display state information being inputted according to the 
partial image information: a control information input unit 
for receiving input of control information which includes a 
name of input signal and display state change command ^ 
instructing to change display state of the partial image 
corresponding to the input signal: a control information 
write unit for writing the inputted control information in a 
second small storage area in the page information storage 
unit: and a page information record unit for successively 35 
recording informadon. which is written as a piece of page 
information in each small storage area in the page informa- 
tion storage unit, in a storage area on the disc. 

According to the above construction, the page informa- 
tion can be stored successively in the storage area on the 40 
disc. Effects mentioned above can be obtained from the 
multi-media information playback device which plays back 
multi -media information from the disc. 

(4> A multi-media information playback device for read- 
ing multi-media information from a data server connected 45 
via a network and playing back the multi-media information 
may comprise a page information read unit for reading a 
piece of page information stored as one file from a data 
server connected via a network, the page information includ- 
ing a partial image list having at least one pair of the partial so 
image information showing a partial image of a screen 
image and display state information describing display state 
of the partial image, and the page informadon also including 
control information which describes a command to be 
executed according to an input signal, the control inforrna- 35 
tion including a display state change command instructing to 
change the display state of the partial image: a display data 
creation unit for creating display data of the screen image by 
placing the partial image shown by the partial image infor- 
mation on a screen: a display unit for displaying each display 6G 
data created by the display data creation unit: an input unit 
for receiving an input signal from an operator, and a 
command execution unit for detecting die input signal 
inputted by the operator, judging whether the input signal 
corresponds to the command described by the control infor- 65 
mation in the page information read by the page information 
read unit, and instructing the display data creation unit to 
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create new display data by changing the display state of the 
corresponding partial image according to the display state 
change command when the input signal corresponds to the 
display state change command: wherein the display data 

5 creation unit may create new display data according to the 
command execution unit. 

According to the above construction, effects mentioned 
above can be obtained when multi-media information is read 
from the data server connected via a network and played 

to back. 

(5) A multi-media information record device for recording 
multi-media information in a data server connected via a 
network may comprise a partial image storage unit for 
storing the partial image information of at least one partial 

15 image in one partial image file, the partial image information 
being a part of the screen image; a partial image list input 
unit for receiving input of a name of at least one partial 
image file in the partial image storage unit and display state 
information showing display state of the partial image 

2C shown by the partial image information in the partial image 
file; a page information storage unit for having a plurality of 
separate small storage areas, each of which is used for 
storing at least a piece of page information: a partial image 
list write unit for writing a pair of the partial image infor- 

25 mation and the display state information in a first small 
storage area in the page information storage unit along with 
the IDs of the partial image information and the display state 
information, after reading partial image information from 
the partial image file whose name is inputted, the display 

30 state information being inputted according to the partial 
image information; a control information input unit for 
receiving input of control information which includes a 
name of input signal and display state change command 
instructing to change display state of the partial image 

35 corresponding to the input signal; a control information 
write unit for writing the inputted control information in a 
second small storage area in the page information storage 
unit; and a page information record unit for recording 
contents of apiece of page information in one file in the data 

40 server after transmitting contents written as a piece of page 
information in each storage area in the page information 
storage unit to a data server along with its write request. 

According to the above construction, each page informa- 
tion can be stored in individual file in the data server, and 

45 above effects can be obtained in the multi-media information 
playback device which reads and plays back multi-media 
information from the data server 

BRIEF DESCRIPTION OF THE DRAWINGS 

50 

These and other objects, advantages and features of the 
invention will become apparent from the following descrip- 
tion thereof taken in conjunction with the accompanying 
drawings which illustrate a specific embodiment of the 

55 invention. In the drawings: 

FIG. 1 is an example of data configuration of multi-media 
information* which is showing synchronous playback of 
images and sound in the conventional multi-media informa- 
tion playback device. 

# FIG. 2 is a block diagram showing functional construction 
of a multi-media informau'on playback device 200 of the 
first embodiment of the present invention. 

FIG. 3 is a block diagram showing hardware construction 
of the multi-media information playback device 200 shown 

65 in FIG. 2, 

FIG. 4 shows how multi-media information is recorded in 
the CD-ROM 201. the multi-media information being 
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played back by the multi-media information playback device 
200 of the present invention. 

FIG. 5 shows data configuration of a page stream played 
back by the multi-media information playback device 200 of 
the present embodiment. 5 

FIG. 6 shows data configuration of time line stream 
played back by the multi-media information playback device 
200 of the present embodiment. 

FIG. 7 is a flow chart showing procedure of playback of iQ 
the multi-media information in the multi-media information 
playback device 200 of the present embodiment. 

FIG. 8 shows contents of the page stream 1. which is a 
concrete example of the page stream shown in FIG. 5. 

FIG. 9(a) shows contents of the initial image iilustrated by 15 
a display data creation unit 2X3 based on the partial image 
list of the page stream 1 shown in FIG. 8. 

FIG. 9(b) shows contents when an input signal "A" is 
inputted while the page stream 1 is executed. 

FIG. 9(c) shows contents right after an input signal U B" in 20 
inputted while the image shown in FIG. 9(b) is displayed. 

FIG. 9(d) shows contents at "00 minute 01.000 second", 
which is an elapsed time after the start of the execution of 
the tune line information. 

FIG. 9{e) shows contents at "00 minute 02.000 seconds", 
which is an elapsed time after the start of the execution of 
the time line information. 

FIG. 9(f) shows contents of the initial image when the 
page stream 2 is loaded to the page storage unit 210. 30 

FIG. 10 shows contents of a time line stream 1. which is 
a concrete example of the time stream shown in FIG. 6. 

FIG. 11 shows contents of a page stream 2. which follows 
the page stream 1 shown in FIG. 8. 

FIG. 12 is a block diagram showing construction of the 35 
multi-media information record device 1200 of the second 
embodiment of the present invention* 

FIG. 13 is a block diagram showing hardware construc- 
tion of the multimedia information 1200 shown in FIG. 12. 40 

FIG. 14 is a Sow chart showing procedure of the record- 
ing of the multi-media information in the multi-media 
information record device 1200 of the present embodiment. 

FIG. 15 is a flow chart showing procedure of the record- 
ing processing of the multi-media information in the multi- 
media information record device 1200 of the present 
embodiment. 

FIG. 16 is a block diagram showing construction of the 
muiti-media information playback device 16W of the third 
embodiment of the present invention. 50 

FIG. 17 is a block diagram showing hardware construc- 
tion of the multimedia information 1600 shown in FIG. 16. 

FIG. 18 is a block diagram showing construction of the 
multi-media information record device 1800 of the fourth 
embodiment of the present invention. 

FIG. 19 is a block diagram showing hardware construc- 
tion of the multimedia information 1800 shown in FIG. 18. 



55 



DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 



60 



(Embodiment 1) | 
FIG. 2 is a block diagram showing functional construction 
of a multi-media information playback device 200 of 
Embodiment 1 of the present invention. FIG. 3 is a block 65 
diagram showing hardware construction of the muiti-media 
information playback device 200 shown in FIG. 2. 
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As shown in FIG. 2. the multi-media information play- 
back device 200 comprises a CD-ROM 201. a CD-ROM 
interface unit 202, an input unit 203. a control information 
execution unit 204. a clock unit 205. a page load unit 206. 

5 a time line control unit 207. a storage unit 208, a display unit 
212. a display data creation unit 213. an audio D/A conver- 
sion unit 214, and an audio output unit 215. The storage unit 
208 further comprises a display data storage unit 209. a page 
storage unit 210. and an audio data storage unit 211. 

10 As shown in FIG. 3. the multi-media information play- 
back device 200 comprises a CPU 301. a RAM 302, a 
system clock 303, a timer 504, a key board 305. a joy pad 
306. an input device interface 307, a disc interface 308, a 
CD-ROM 309, a MPEG decoder 31L a sound D/A convenor 

15 312, a speaker 313. a graphic controller 314, a VRAM 315. 
an image D/A convenor 316. and a CRT 317. as components 
of hardware. The CPU 301 comprises functions to control 
operation of each component of the multi-media information 
playback device 200, in addition to the functions which are 

20 explained as follow. 

The CD-ROM 201 can be attached/detached to the multi- 
media information playback device 200. The following 
multi-media information is written by High Sierra Format of 
ISO 9690 and the like. 

FIG. 4 shows how multi-media information, which is 
played back by the multi-media information playback device 

200 of the present embodiment, is stored in the CD-ROM 
201. Storage areas 401. 402. and 403 of the CD-ROM 201 

3Q continues successively on the track, and data stored in each 
of the storage areas is read one after another as the CD-ROM 

201 turns. A page stream 1. which is mainly composed of 
partial image data and mentioned later, is recorded in the 
storage area 401. A page stream 2. which is mainly com- 

35 posed of partial image data and shows the page following the 
page stream 1. is recorded in the storage area 402. A time 
line stream 1. which is mainly composed of data about 
dynamic pictures and animations relating to the page stream 
1, is recorded in the storage area 403, and mentioned later. 

^ Data is read from the CD-ROM 201 while the page stream 
1, the page stream 2. and the time line scream 1 are used as 
a logical unit, respectively. But they don't necessarily cor- 
respond to the physical unit of the CD-ROM 201. like a 
sector. 

4 5 A page is an image to be displayed on the screen, which 
is displayed every l /so second or so. Each page stream is 
multi-media information which describes contents of a plu- 
rality of pages. Each of the pages is composed of one or a 
plurality of predetermined partial images. If the original 

50 partial image is the same as the partial image which is 
mentioned later by the page stream, a partial image can be 
enlarged/reduced or can be moved parallel/rotationaiiy and 
one or a plurality of partial images can be deleted by the 
same page stream. 

55 FIG. 5 shows data configuration of a page stream which 
is played back by the multi-media information playback 
device 200 of the present embodiment. A page stream 
consists of a partial image list, control information, and time 
line information. Each page stream is given an identification 

$o number. 

In the partial image list, contents of each partial image 
consisting of one screen image is written: a partial image ID 
which designates a partial image; X-Y coordinates which 
show the initial position of the partial image when it is 
65 displayed in the page; X-Y enlargement rates which show 
the scale of enlargement or reduction when the partial image 
is first displayed in the page; a rotational angle between a 
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partial image shown by the partial image data and the partial 
image displayed in the page; and partial image data which is 
expressed by the values corresponding to the pixels of the 
partial image. The partial image ID is expressed by the 
number which shows order of placing the partial images in 5 
the display data, when display data of one screen image is 
created by the partial images shown by the page stream. 
Therefore, when the display areas of the partial image 
partially overlap with each other, the partial image data 
corresponding to the pixels of the overlapping areas is 10 
updated by partial image data which has a greater partial 
image ID. So partial images axe displayed according to the 
partial image ID. which means that the partial image having 
a greater partial image ID covers the prodding partial 
images. The partial image data shown by the page stream 15 
can be expressed by compressed and decoded bit map data 
and. if the partial image is geometrical graphic form, it can 
be expressed by graphic data. Also, the partial image does 
not necessarily have to be a graphic form. It can be a string 
showing an input request displayed on each menu screen. In 20 
this case, the string can be shown by a character code. 

The control information describes commands to be 
executed by the multi-media information playback device 
200 for the input signal from the input unit 203 while the 
page shown by the page stream is displayed. The commands 25 
are described for each input signal. The commands are: 
change the X-Y coordinates of the partial image; change the 
X-Y enlargement rates of the partial image; change the 
rotational angle of the partial image: load other page 
streams; start the execution of the time line information; 3C 
overlap a plurality of partial images; display the color 
images in black and white: emphasize a specific color; 
change frequency of sound: and the like. 

The time line information describes animation display and 
dynamic picture display; the animation display moves pre- 35 
determined image pictures shown by the partial image list as 
time goes by; the dynamic picture display displays the 
dynamic pictures shown by the time line stream mentioned 
later. Specifically, as for the animation display, the following 
items are shown: a partial image ID of the partial image to 40 
be displayed in animation (animation object); X-Y coordi- 
nates which shows a predetermined display position at every 
given time after the start of the animation display of the 
partial image; and an animation display end time of the 
partial image. As for the dynamic picture display, the fol- 45 
lowing items are shown: the identification number of the 
time line stream which shows images to be displayed in 
dynamic picture; and a start time and an end time of the 
dynamic picture display. Also, when the dynamic picture 
image is displayed inside the predetermined partial image 50 
shown by the partial image list, the partial image ID of the 
partial image which is displayed in dynamic picture is 
shown so that the dynamic picture is displayed in place of 
the predetermined partial image. The dynamic picture image 
shown by the time line stream mentioned later does not 55 
necessarily have to be displayed inside the predetermined 
partial image shown by the partial image list 

FIG. 6 shows data configuration of the rime line stream 
which is played back by the multi-media information play- 
back device 200 of the present embodiment. Identification 60 
numbers are given to each time line stream. As described 
already, dynamic pictures and ankiatioas are displayed by 
playback of the successive still pictures at every display 
time, whicn is every second or so. In the time line scream 
shown in FIG. 6. still picture data expressed by the com- 65 
pressed and encoded bit map data is stored according to the 
order of Che display, along with the audio data corresponding 
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to the still picture. Also the audio data is compressed, 
encoded, divided into predetermined equal sizes correspond- 
ing to the display time of the stUl picture data, and played 
back successively while the still picture data corresponding 
5 to the audio data is displayed. Each display time can be 
expressed by the elapsed time after the start of the clock unit 
205. When the clock 205 can measure the elapsed time from 
the start of the display for a pluraiiry of time streams, the 
display time can be expressed by the e.apsed time from the 
io start of the execution of each time line stream. 

Also the audio data and the still picture data do not 
necessarily have to be shown by the time line stream. For 
example, when the audio data shown by the time line stream 
is played back while the display picture is displayed by the 
15 partial image shown by the page stream, only audio data can 
be shown in the time line stream. Animations as well as 
dynamic pictures can be shown by this time line stream. So 
in order to distinguish these animations from the animations 
displayed by the partial image shown by the page stream, a 
20 term "dynamic picture display" is used for the display of 
animations and dynamic pictures using the still picture data 
shown in the time line stream. 

The CD-ROM interface unit 202 is implemented by the 
disc interface 308. and reads multi-media information into 
25 the multi-media information playback device 200 according 
to the instruction of die page load unit 206. the multi-media 
information being described by the page stream and the time 
line stream recorded in the CD-ROM 201. 
The input unit 203 comprises the joy pad 306. the key 
30 board 305. and operation unit like a mouse, which is not 
illustrated, and receives input of instruction from a user via 
the input device interface 307. The input unit 203 outputs the 
input signal corresponding to the received input. For 
example, this unit 203 outputs the input signals A. B. and C 
35 which correspond to the buttons A. B. and C in the joy pad 
306. to the control information execution unit 204. 

The control information execution unit 204 is imple- 
mented by the CPU 301. This unit 204 interprets and 
^ interactively executes control information shown by the 
page stream loaded to the page storage unit 210. Specifically, 
the control information execution unit 204 instructs the page 
load unit 206. the time line control unit 207. and the display 
data creation unit 213 to execute processing based on the 
45 instruction of the control information. 

The clock unit 205 is implemented by the system clock 
303 and the timer 304, Being activated by the time line 
control unit 207. the timer 304 measures the elapsed time 
from the execution start of the time line information with 
50 reference to the system clock 303. The clock unit 205 can be 
equipped with a plurality of the timers 304 so that the 
elapsed time from the start of the execution of each time line 
stream can be measured along with the elapsed time from the 
start of the execution of the time line information. 
55 The page load unit 206 is implemented by the CPU 301. 
This unit 206 instructs the CD-ROM interface unit 202 to 
read out a predetermined page stream and loads the page 
stream to the page storage unit 210. when the multi-media 
information playback device 200 is activated. This unit also 
60 instructs the CD-ROM interface unit 202 to read the desig- 
nated page stream and the time line stream according to the 
instruction of the time line control unit 207 or the control 
information execution unit 204, and loads the page stream or 
the time line stream to the page storage unit 210. 
65 The time line control unit 207 is implemented by the CPU 
301. and starts the execution of the time line information 
described by the loaded page stream according to the control 
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Information execution unit 204, This unit 207 activates the 
clock unit 205 when the execution of the time line infor- 
mation is started. This unit 207 initializes the timer 304 of 
the clock unit 205 when the execution of the time line 
information is completed. In case each time line stream is 5 
equipped with the timer 304 which measures the elapsed 
time from the start of the execution, corresponding tinier 304 
is initialized when execution of each time line stream is 
completed. 

The time line control unit 207 instructs the display data ic 
creation unit 213 to place designated partial images, carry- 
ing out the designated processing like enlarging/reducing 
and rotational migrating, at the designated position of the 
X-Y coordinates, according to the elapsed time from the start 
of the execution of the time line information measured by i$ 
the clock unit 205 when the time line information which 
instructs animation display is executed. 

The time line control unit 207 instructs the page load unit 
20$ to load the time line stream designated in the time line 
information to the page storage unit 210, when the time line 20 
information which instructs the dynamic picture display is 
executed. This unit 207 designates the still picture data 
corresponding to the elapsed time from the start of the 
execution of the time line information measured by the clock 
unit 205 and posts the designated still picture data to the 25 
display data creation unit 213. Then this unit 207 instructs 
the display data creation unit 213 to place the still picture 
data at the predetermined position, for example, inside the 
partial image designated in the time line information* At the 
same time, this unit 207 reads the audio data corresponding 30 
to the still picture data from the time line stream, and 
transfers the audio data to the audio data storage unit 21L 

The display data storage unit 209 is implemented by the 
VRAM 315. and stores the display data of one screen page 
which is created by the display data creation unit 213. 35 

The page storage unit 210 is implemented by the RAM 
302. and stores the page stream and the time line stream read 
from the CD-ROM 201. 

The audio data storage' unit 211 is implemented by the 40 
RAM 302. and stores audio data which is divided into about 
Vx second sizes. 

The display unit 212 is implemented by the image D/A 
converter device 316 and the CRT 317. This unit 212 reads 
the display data stored in the display data storage unit 209 45 
every Vx> second or so and displays it in the CRT. The image 
D/A converter 316 reads the display data from the VRAM 
315, which is the display data storage unit 209. every l /3o 
second or so. converts the display data to analog signals, and 
supplies them to the CRT 317. The CRT 317 displays the 50 
analog signals which are converted from the display data. In 
the present embodiment the CRT 317 is used for the display 
unit 212. but a liquid crystal display LCD or a television 
which is connected to the multi-media information playback 
device 200 can substitute for the CRT 317. The contents of 55 
the display unit 212 can be printed on papers. 

The display data creation unit 213 is implemented by the 
CPU 301, the MPEG decoder 311. and the graphic controller 
314. This unit 213 creates display data of a whole screen of 
an initial image, using the partial images shown by the 60 
loaded page stream, when a new page steam is loaded to the 
page storage unit 210. According to the instruction of the 
control information execution unit 204. this unit 213 carries 
out the designated processing {e.g. moving X-Y coordinates, 
enlarging/reducing, rotational moving, etc.) for the partial 65 
images shown by the loaded page stream, and creates 
display data of a whole screen which shows a still image 
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using the partial Images. According to the instruction of the 
time Line control unit 207. the unit 213 carries out the 
designated processing mentioned above for the partial 
images shown by the loaded page stream, creates display 
5 data for animation display, and update the display data for 
dynamic picture display, using the still picture data desig- 
nated by the time line control unit 207 from the still picture 
data shown by the designated time line stream. At this point, 
the MPEG decoder 311 expands and decodes the image data. 
ic which was compressed and encoded according to the 
MPEG, such as still picture data shown by the time line 
stream and partial image data shown by the page stream. 

The audio D/A converter unit 214 is implemented by the 
MPEG decoder 311 successively and the sound D/A con- 
is vertex 312. The MPEG decoder 311 reads the audio data 
stored in the audio data storage unit 211. and expands and 
decodes the audio data, which is stored compressed and 
decoded according to the MPEG. The sound D/A converter 

312 converts the decoded audio data into analog signals, and 
20 outputs them to the speaker 313. As well as the image data. 

the audio data does not necessarily have to be compressed 
and decoded according to the MPEG. The audio data can be 
compressed and decoded according to other formats, or does 
not have to be compressed and decoded at all. Therefore, the 
25 MPEG decoder 311 can be substituted for other decoders. 
When audio data which is not compressed and decoded is 
treated, there's no problem without decoders. 
The audio output unit 215 is implemented by the speaker 

313 and converts analog signals from the audio D/A con- 
30 verter unit 214 to sound and outputs the sound. 

Explanation about the procedure of the playback of multi- 
media information in the multi-media information playback 
device 200 follows next: it refers to a flow chart in FIG. 7; 
concrete examples of the page stream 1. the page stream 2 

35 and the time line stream illustrated from FIGS. 8-11; and 
concrete example of the contents displayed based on them, 
FIG. 7 is a flow chart showing the procedure of the 
playback of multi-media information in the multi-media 

^ information playback device 200 of the present embodi- 
ment. 

The page load unit 206 instructs the CD-ROM interface 
unit 202 to read a predetermined page stream, like page 
stream 1 mentioned later, from the CD-ROM 201, and loads 
45 the read page stream to the page stream storage unit 210, 
(Step 701). 

The display data creation unit 213 creates display data of 
the initial image from each partial image shown by the 
partial image list of the loaded page stream (Step 702). The 

50 display data is transferred to the display data storage unit 
209. stored there, read and displayed by die display unit 212 
every l /x> second or so. 

FIG* 8 shows contents shown by the page stream 1. which 
is a concrete example of the page stream shown in FIG. 5. 

55 In the partial image list, a circle having a pattern of oblique 
lines inside, a triangle having a partem of vertical dashed 
lines inside, and a rectangle having no pattern inside are 
shown by the bit map data or graphic data as partial image 
data, according to the order of the partial image IDs. Also the 

60 X-Y coordinates which shows the display position of the 
partial image, the X-Y enlargement rates of the partial 
image, and a rotational angle arc shown. In case of a circle 
whose partial image ID is 1, bit map data shown by the 
partial image data is placed based on the X-Y coordinates 

65 ( 100. 100) by the display data creation unit 213. and display 
data is created with the enlargement rates being (1.1). and 
the rotational angle being 0. The display data of the triangle 
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and the rectangle whose partial image IDs are 2 and 3, 
respectively, is created in the same way mentioned above. So 
in the areas where each partial image overlaps, the partial 
image data having a small partial image ID is updated by the 
partial image data having a greater partial image ID, 5 

FIG. x Ka) shows the contents of the initial image, which 
is illustrated by the display data creation unit 213. based on 
the partial image list of the page stream 1 illustrated in FIG. 
8. As mentioned already, in the illustration processing of the 
display data creation unit 213, the partial image data having W 
a smaller partial image ID is updated by the partial image 
having a greater partial image ID in areas where each partial 
image overlaps. So. as illustrated in the initial image of the 
FIG. 9(a). partial images having a greater partial image ID 
are displayed on top of the layers of other partial images. 15 

The control information execution unit 204 detects an 
input signal from the input unit 203. when the processing of 
Step 702 is completed (Step 703). and starts Step 708 when 
the input signal is not detected. 

When the input signal is detected, the control information 
execution unit 204 identifies the input signal (Step 704). and 
judges whether the input signal is shown by the control 
information of the page stream (Step 705). When the input 
signal is not shown, the control information execution unit 
204 starts Step 708. 

The control information execution unit 204 executes the 
command which is described according to the input signal, 
when the identified input signal is judged to be shown by the - 
control information of the page stream (Step 706). The 3Q 
display data creation unit 213 creates display data based on 
the execution result of the control information execution unit 
204 (Step 707). 

The following explanation is about the processing from 
Steps 705-707 in the multi-media information playback 35 
device 200, referring to concrete examples. 

The control information execution unit 204 identifies an 
input signal, and reads the control information from the page 
stream 1 in FIG. 8, which is loaded to the page storage unit 
210. The control information is shown as A, B, and C, so that 40 
the control information execution unit 204 can judge the 
type of the input signal. 

The control information execution unit 204 executes a 
command *The X-Y coordinates of the partial image 1 
should be ( 170. lOOf . That is because when the identified 45 
input signal is A. u is shown by the control information. 
Specifically, the control information execution unit 204 
passes parameters "partial image 10=1". and "X-Y coordi- 
nates (170, 100r to the display data creation unit 213. The 
control information execution unit 204 instructs the display 5c 
data creation unit 213 to create display data based on the 
parameters. The display data creation unit 213 creates the 
display data according to the instruction, FIG. 9{b) shows 
contents when input signal A is inputted while the page 
stream 1 is executed. The partial image 1 is moved from 55 
(100. 100) to (170. 100). 

When the identified input signal is B. the control infor- 
mation execution unit 204 executes a command 'The clock 1 
of the time line should be started.*', which corresponds to the 
input signa. B. Specifically, the control information execu- 6C 
tion unit 204 instructs the time line control unit 207 to start 
the execution of the time line information shown by the page 
stream 1. Then the time line control unit 207 activates the 
clock unit 205. The time line control unit 207 also reads the 
time line information by the page stream 1. and when there's 65 
is a designated time line stream in the time line information 
which shows the dynamic picture display, the time line 
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control unit 207 instructs the page load unit 206 to load the 
time line stream. The time line control unit 207 further 
creates parameters mentioned later based on the time line 
information, and passes it to the display data creation unit 

5 213. and instructs the display data creation unit 213 to create 
displav data based on the parameters- 

As is shown in FIG. 8. time line information about the 
animation display and the dynamic picture display is shown 
by the page stream 1. 

10 The following items are shown concerning the animation 
display: a partial image ID "3 M of the partial image to be 
displayed in animation (animation subject); 00 minute 
00.000 second as an animation display start time and 00 
minute 03.000 seconds as an animation display end time. 

15 both of which are expressed by the elapsed times from the 
start of the execution of the time line information: X-Y 
coordinates (150. 150) at 00 minute 00.000 seconds, X-Y 
coordinates (170. 170) at 00 minute 01.000 seconds, X-Y 
coordinates ( 190, 150) at 00 minute 02.000 seconds, all of 

2C which show the display positions of the partial image 3 at 
predetermined elapsed times. 

The following items are shown concerning the dynamic 
picture display: an identification number 'T* of the time line 
stream to be displayed in dynamic pictures; 00 minute 

25 00,000 second as a dynamic picture display start time and 00 
minute 03.000 seconds as dynamic picture display end time, 
both of which are expressed by the elapsed times from the 

. start of the execution of the time line information, 

30 The time line control unit 207 instructs the page load unit 
206 to load the time line stream 1 to the page storage unit 
210. FIG. 10 shows contents of the time line stream 1. which 
is a concrete example of the time line stream shown in FIG. 
6. Still picture data which shows the rising sun and audio 

35 data to be played back while the still picture data is dis- 
played are shown by the time line stream, both of which 
correspond to each elapsed time between 00 minute 00.000 
second and 00 minute 03.000 seconds. 
When the identified input signal is C. the control iofor- 

45 mation execution unit 204 executes a command **Read the 
page stream 2*\ which corresponds to the input signal C. 
Specifically, the identified input signal instructs the page 
load unit 206 to load the page stream 2 to the page storage 
unit 210. Then the page load unit 206 instructs the CD-ROM 

4 5 interface unit 202 to read the page stream 2, and loads the 
read page stream 2 to the page storage unit 210. The display 
data creation unit 213 creates display data based on the 
partial image list of the loaded page stream 2. 
FIG. 11 shows contents of the page stream 2. which 

5c follows after the page stream 1 shown in FIG. 8. In FIG. 11, 
a rectangle having a pattern of oblique lines inside is shown 
by the partial image data as bit map data, along with a partial 
image data ID *T\ X-Y coordinates (100. 100), X-Y 
enlargement raxes (LI), a rotational angle *'0*\ Also, a 

55 command for the input signal A 'The X-Y* coordinates of the 
partial image I should be (200. 200)" is described by the 
control information of the page stream 2. Therefore, when A 
is inputted while the page stream 2 is executed based on the 
partial image list mentioned above, the partial image 1 is 

6c placed at (200. 200) of the X-Y coordinates. A time line 
information is not shown by the page stream 2. 

FIG. 9, (/) shows contents of the initial image when the 
page stream 2 is loaded to the page storage unit 210. The 
rectangle having oblique lines inside, which is shown by the 

65 partial image list of the page stream 2. is shown by X-Y 
enlargement rates (Li), a rotational angle "0*\ and X-Y 
coordinates {100. 100). 
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The time One control unit 207 checks the clock unit 205 
to see whether it is operating, when Step 707 finishes (Step 
70S). When the clock unit 205 is operating, the time line 
control unit 207 reads the elapsed time from the start of the 
execution of the time line information, which is measured by 5 
the clock unit 205 (Step 709), 

The time line control unit 207 judges whether there is a 
partial image designated to be displayed in animation, by the 
elapsed time read from the clock unit 205 (Step 710). When 
there is such a partial image, the time line control unit 207 10 
calculates X-Y coordinates, which shows the display posi- 
tion of the partial image, from the already designated X-Y 
coordinates which correspond to a predetermined elapsed 
time. Then the time line control unit 207 passes the calcu- 
lated X-Y coordinates. X-Y enlargement rates, a rotational * 5 ' 
angle and a partial image ID of the partial image to the ; 
display data creation unit 213 as parameters. Then the time \ 
line control unit 207 instructs the display data creation unit 
213 to create display data based on the parameters. The 
display data creation unit 213 creates display data based on 20 
the parameters, and updates the display data in the display 
data storage unit 209 ("Step 711). When there's no designated 
partial image in Step 710, the time line control unit 207 starts 
Step 712. 

For example, when the elapsed time in Step 709 is 00 25 
minute 00.000 second, the animation display start time 
shown by the time line information is 00 minute 00.000 
second. So the time line control unit 207 creates a partial 
image ID u 3" and X-Y coordinates (150.150)) at 00 minute 
00.000 second as parameters for animation display, and 30 
passes them to the display data creation unit 213. Then the 
display data creation unit 213 moves the partial image 3 to 
the position designated by the parameter "the X-Y coordi- 
nates ( 150.150) at 00 minute 00.000 second", creates display 
data, and updates the display data in the display data storage 35 
unit 209. 

Also, when an elapsed time is 00 minute 0L000 second in 
Step 709. the partial image 3. which is designated to be 
displayed in animation, exists. So the time line control unit ^ 
207 creates an animation display partial image ID "3 M . and 
X-Y coordinates (170.170) at 00 minute 01.000 second as 
parameters for animation display. Then this unit 207 passes 
them to the display data creation unit 213. The display data 
creation unit 213 moves the partial image 3 to (170, 170) 45 
based on the parameters and creates display data, and 
updates the display data in the display data storage unit 209. 

When the elapsed time is 00 minute 00.500 second, the 
time line control unit 207 calculates **X-Y coordinates 
(160.160) at 00 minute 00.500 second" which is interpolat- 50 
ing between the X-Y coordinates (150,150) at 00 minute 
01.000 second and X-Y coordinates (170. 170) at 00 minute 
01.000 second, and passes it to the display data creation unit 
213. The display data creation unit 213 moves the partial 
image 10 the (160. 160), creates display data and updates the 55 
display data in the display data storage unit 209. 

Moreover, the time line control unit 207 judges whether 
there is a time line stream designating dynamic picture 
display . by the elapsed time read from the clock 205 (Step 
712). When the designated time line stream exits, the time 60 
line control unit 207 designates the stUl picture data corre- 
sponding to the elapsed time read from the clock unit 205, 
and posts the data to the display data creation unit 213. Also 
the time line control unit 207 reads audio data corresponding 
to the still picture data from the time line stream and €5 
transfers that data to the audio data storage unit 211. The 
display data creation unit 213 reads the designated stM 
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picture data from the designated time line stream and creates 
display data. After the time line control unit 207 updates the 
display data in the display data storage unit 209 (Step 713). 
this unit starts Step 703. If the designated time line stream 
5 does not exist in Step 712. the time line control unit 207 
starts Step 703. 

For example, when the elapsed time in Step 709 is 00 
minute 00,000 second, the dynamic picture display start time 
shown by the time line information is 00 minute 00.000 
L0 second. So the time line control unit 207 creates identifica- 
tion number "1" of the time line stream. 00 minute 00.000 
second of the still picture data, and the partial image ID 4fc 3 M 
as parameter for the dynamic picture display corresponding 
to the elapsed time 00 minute 00.000 second from the start 
l5 of the execution of the time line information, and passes 
them to the display data creation unit 213. At the same time, 
the time line control unit 207 reads audio data Al from the 
time line stream 1. and transfers it to the audio data storage 
unit 211. The time line control unit 207 instructs the display 
lQ data creation unit 213 to create display data based on the 
parameter. Then the display data creation unit 213 places the 
still picture data designated by the parameter "still picture 
data=00 minute 00.000 second" inside partial image 3. 
creates display data, and updates the display data in the 
25 display data storage unit 209. 

FIG. 9(c) shows contents right after the input signal B is 
inserted while FIG. 9(b) is displayed. FIG. 9(c) shows a 
screen image created based on each parameter whose 
elapsed time measured by the clock unit 205 is 00 minute 
3C 00.000 second. The partial image 3 to be displayed in 
animation is displayed by "X-Y coordinates (150. 150) at 00 
minute 00.000 second", while the partial images 1 and 2 are 
displayed on the same position as FIG. 9(b). since their X-Y 
coordinates are not designated by parameters for animation 
35 display. Also, still picture data of "still picture data=00 
minute 00.000 second" in the time line stream 1 is displayed 
inside the partial image 3. At the same time, the audio data 
Al is played back. The difference between the contents of 
the initial image shown in FIG. 9(a) and that of FIG. 9(c) is 
40 the position of the circle having a pattern of oblique lines. In 
the former case, the circle is shown by the X-Y coordinates 
(100. 100); in the latter case, it is displayed by (170. 100). 

Moreover, when the elapsed time in Step 709 is 00 minute 
01.000 second, the time line stream 1 designating dynamic 
45 picture display, exists. So the time line control unit 207 
creates parameters "identification number of the time line 
stream=r. "still picture data=00 minute 01.000 second", 
and "partial image ID=3" corresponding to the elapsed time 
"00 minute 01.000 second" for dynamic picture display, and 
so passes them to the display data creation unit 213. At the 
same time, the time line control unit 207 reads the audio data 
Ai corresponding to the elapsed time 00 minute 01.000 
second, from the time line stream 1. and transfers the data to 
the audio data storage unit 211. The display data creation 
55 unit 213 places the still picture data of the "00 minute 01.000 
second", which is read from the time line stream 1. inside the 
partial image 3 according to the parameter, and creates 
display data. 

FIG. 9(<*> shows contents at 00 minute 01.000 second. 

60 which is the elapsed rime from the start of the execution of 
the time line information. As shown in FIG. 9(d). the partial 
images 1 and 2 are displayed at the same positions as FIG. 
9(b). but the partial image 3 is moved to the position of the 
X=Y coordinates (170. 170). and the still picture data of DO 

65 minute 01.000 second" is displayed iasidt the partial image 
3. While this image is displayed, the audio data Ai is played 
back. 
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Moreover. FIG. 9 shows contents at the elapsed time. 00 
minute 02.000 seconds. The partial images 1 and 2 are 
displayed at the same positions as they are displayed in FIG. 
9{e). by the same processing mentioned above. The partial 
image 3 is move to (190. 150), and a still picture data of "00 5 
minute 02.000 seconds'* are displayed inside it. While this 
still picture is displayed, the audio data Aj is played back. 
X-Y coordinates for animation display for the partial image 
3 after "00 minute 02.000 seconds'* is not shown by the 
control information in the page stream 1. While animation 10 
display continues until "00 minute 03.000 seconds*', the 
partial image 3 is displayed at the X-Y coordinates (190. 
150). However, still picture data until M 00 minute 03.000 
seconds*' are successively displayed inside the partial image 
3. and the audio data is also played back at the same time. 15 
As explained above, display of partial images can be ! 
changed easily according to the present embodiment, com- '; 
pared to the case in which contents of one screen is stored 
as one image data in the recording medium. Also, even 
though the contents are limited to the combinations of the 
partial images, maximum number of the still pictures to be 
displayed can be increased regardless of the storage capacity 
of the recording medium. In case of changing the display of 
the partial image shown by in the loaded page stream, as the 
image data does not have to be read from the recording 25 
medium, the multi-media information playback device 200 
can display many still pictures per unit time, regardless of 
the transfer rate. The contents of each page stream and each 
time line stream is successively stored on the same track of 
the CD-ROM. so head seek does not happen when the 30 
information is successively read. Speed of data readout from 
the same page stream and the same time line stream is fast 
and breaks in the sound and dynamic pictures do not happen, 
so that the multi-media information is played back accu- 
rately. As the partial image list, the control information 35 
relating to the partial image, and the time line information 
are described by one page stream, the image data, control 
information, and dynamic information are loaded to the 
RAM 302 and the load of the CPU is decreased, unlike the 
case in which the information are recorded in the different 40 
storage areas in the recording medium. 

In the present embodiment, the still picture data and the 
audio data shown by the time line stream are compressed 
and encoded according the MPEG, and they are expanded 
and decoded by the MPEG decoder 311. But the still picture 45 
data, the audio data and the partial image data are not 
necessarily have to be compressed and encoded according to 
the MPEG. They can be compressed and encoded by other 
formats like VFW (Video for Windows) or QT (Quick 
Time). Therefore the MPEG decoder 311 can be substituted 5c 
for decoders following other formats, or a decoder is not 
necessary at all in case only the data which is not com- 
pressed and encoded is dealt with. 

In the present embodiment, the page load unit 206 loads 
the contents of the designated time line stream from the 55 
CD-ROM 201 to the page storage unit 210 at a time, when 
the execution of the rime line information is started: the 
display data creation unit 213 reads the designated still 
picture data in the loaded rime line stream and creates 
display data. However, the contents of the time line stream 60 
does not necessarily have to be loaded at a time. For 
example, the page load unit 206 can load the designated still 
picture data and audio data in the designated timed line 
stream to the page storage unit 210 at each display time of 
the dynamic picture display, and the display data creation 65 
unit 213 can read the loaded still picture data from the page 
storage unit 210 and create display data, and the time line 
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control unit 207 can transfer the loaded audio data to the 
audio data storage unit 211. Generally, the amount of the 
compressed and encoded image data is large, so the data 
amount shown by one lime line stream can be considered 

5 rather large. Therefore, considering the storage capacity of 
the RAM 302. which is necessary as the page storage unit 
210, the page toad unit 206 had better load the designated 
still picture data and audio data to the page storage unit 210 
at each display time of the dynamic picture display. As the 

id contents of the time line stream is successively stored on the 
same track, seek of a read head does not happen, even if the 
designated sti]] picture data and audio data are loaded from 
the CD-ROM 201 at each display time. This enables dis- 
playing of dynamic pictures without breaks in sound and 

15 images. 

In the present embodiment, the time line control unit 207 
instructs the page load unit 206 to load the time line stream 
designated in the time line information in time of dynamic 
picture display; the page load unit 206 instructs the 

20 CD-ROM interface stream unit 202 to read the time line 
stream from the CD-ROM 201. But the time line stream 
does not necessarily have to be read in this way. and the time 
line control unit 207 can directly instruct the CD-ROM 
interface unit 202 to read the designated time line stream. 

25 In the present invention, the display unit 212 displays 
each page every Vx> second or so. But in order to display 
dynamic picture images of high quality, each page has to be 
displayed every Vfeo or Viso second or so. In this case, still 
picture data and audio data corresponding to the display time 

30 of every ! /<so second or Viso second from the start of the 
execution of the time line stream are shown. 

In FIG. 4. the page stream and the time line stream 
correlating with each other, are stored successively in the 
storage areas on the track of the CD-ROM 20L The most 

35 important point is that contents of each page stream or each 
time line stream is stored successively on the same track: the 
contents of the page stream and the time line stream do not 
have to be correlating with each other. For example, only 
one of the page stream and the time line stream can be stored 

4(3 successively in the storage areas. 

In the present embodiment, the CD-ROM 201 is 
explained as the recording medium, but other storage 
medium like DVD can substitute for the CD-ROM 201. 

45 Broader interpretation is possible: the recording medium can 
be a transmission line like a communication line, or a 
storage device of a video server mentioned later. In the 
storage area, all information does not have to be stored in the 
form of digital data. Part of the data. e.g. audio data of the 

^ time line stream, can be analog data and can be stored 
divided. In this cased, the multi-media information playback 
device 200 can be equipped with a storage unit of analog 
data instead of the audio data storage unit 211. 
{Embodiment 2) 

55 FIG. 12 is a block diagram of construction of a multi- 
media information record device 1200 of the second 
embodiment of the present invention. FIG. 13 is a block 
diagram of the hardware construction of the multi-media 
information record device 1200 shown in FIG. 12. 

60 As shown in FIG. 12. the multi-media information record 
device 1200 comprises an input unit 1201. a stream creation 
unit 1202. a second storage unit 1203. a CD-ROM write unit 
1209 and a CD-ROM 1210. The second storage unit 1203 
comprises a partial image storage unit 1204. a dynamic 

65 picture storage unit 1205. an audio storage unit 1206. a page 
stream storage unit 1207 and a time line stream storage unit 
1208. 
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As shown in FIG. 13. elements of the hardware construc- 
tion of the multi-media information record device 1200 is 
almost the same as the multi-media information playback 
device 200 shown in FIG. 3. except that the multi-media 
information record device 1200 does not comprise the joy 5 
pad 306 but comprises a hard disc 1310. 

The input unit 1201 is implemented by a key board 1305 
and an input device interface 1307. The input unit 1201 
comprises a scanner and a video capture board, both of 
which are not illustrated. The scanner inputs printed still 10 
pictures as digitaiized still picture data. The video capture 
board inputs motion pictures from a connected video tape 
recorder as digitaiized dynamic picture data. 

The stream creation unit 1202 is implemented by a CPU 
1301. According to the processing procedure, this unit 1202 ^ 
instructs a CRT 1317 to display a menu screen and an input 
request, and receives input of instruction and character 
information from an editor of multi-media information. 
According to the instruction, the stream creation unit 1202 
reads a partial image file, a dynamic picture file and audio 20 
file corresponding to each instruction, from the partial image 
storage unit 1204, the dynamic picture storage unit 1205. 
and the audio storage unit 1206. respectively, and interac- 
tively creates the page stream and the time line stream 
shown in FIGS. 5 and 6. Input of the control information is 25 
carried out by a character input from the input unit 1201. The 
processing of the stream creation unit 1202 is explained 
later, by means of flow charts in FIGS. 14 and 15. 

The second storage unit 1203 is implemented by a hard 
disc 1310. This unit comprises the partial image storage unit 30 
1204. the motion picture storage unit 1205, the audio storage 
unit 1206. the page stream storage unit 1207. and the time 
line stream storage unit 1208. 

The partial image storage unit 1204 stores one or a 
plurality of partial image filers). One partial image is already 35 
cut out from material data for expressing a stRl picture. The 
partial image is shown by bit map data or graphic data which 
are compressed and encoded according to MPEG, and stored 
in each partial image file. Contents in each partial image file 
correspond to the partial image data of the page stream. 40 
When the material data is correlated with control informa- 
tion or other material data by editing, it constitutes multi- 
media information. 

The dynamic picture storage unit 1205 stores one or a 
plurality of dynamic picture fiie(s). In the dynamic picture 
file, still picture data cut out from the material data of still 
pictures for expressing dynamic pictures are stored in order 
of the display time, corresponding to a predetermined dis- 
play time. 5c 

The audio storage unit 1206 stores one or a plurality of 
audio file{s). Audio data divided into the amount of sound 
data, which corresponds to the length of time in which one 
stUi picture data is displayed, are stored in order of the 
display time. 55 

The page stream storage unit 1207 stores the page stream 
created by the stream creation unit 1202. 

The time line stream storage unit 120S stores the time line j 
stream created by the stream creadon unit 1202. j 

The CD-ROM write unit 1209 is implemented by a disc ec | 
interface 1308. According to the instruction of the input unit I 
1201. this unit 1209 reads the page stream and the time line J 
stream from the page stream storage unit 1207 and the time ' 
line stream storage unit 1208. respectively, and stores them ! 
in the CD-ROM 1110. 65 ! 

FIGS. 14 and 15 show flow charts showing procedure of 
storing multi-media information in the multi-media infor- 
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mation record device 1200 of the present invention. Steps 
1419-142$ are shown in a flow chart in FIG. 15. 

The stream creation unit 1202 initializes the identification 
number of the page stream and the time line steam to "i". 

5 (Step 1401) 

The stream creation unit 1202 requests an input instruct- 
ing whether contents to be described exist in the page 
stream. When the contents to be described exist in the page 
stream (Step 1402). the stream creation unit 1202 initializes 

10 the partial image ED "1" (Step 1403). 

The stream creation unit 1202 further requests input 
instructing whether contents to be described as a partial 
image of the partial image ID exist. When the content exist 

15 (Step 1404;. the stream creation unit 1202 receives a name 
of the partial image file which shows the partial images (Step 
1405). Also, the stream creation unit 1202 receives input of 
X-Y coordinates, X-Y enlargement rates and a rotational 
angle, each of which describing the display position, 
enlargement rate and rotational angle of the partial image on 
the initial screen of the page stream (Step 1406). The stream 
creation unit 1202 writes the partial image ID. X-Y 
coordinates. X-Y enlargement rates and the rotational angle 
in the storage areas allocated to each item of the partial 
image ID of the page stream in the page stream storage unit 

* 1207. Then the stream creation unit 1202 reads the partial 
image file, whose name is inputted, from the partial image 
storage unit 1204. and writes the contents of the file in the 
storage areas allocated to each item of the partial image data 
having the partial image ID of the page stream in the page 
stream storage unit 1207 (Step 1407). Then, the stream 
creation unit 1202 increments the partial image 3D (Step 
1408). and goes back to Step 1404. 
In Step 1404. when the contents to be described as the 

3 5 partial image of the partial image ID does not exist, the 
stream creation unit 1202 further requests input instructing 
whether the page stream has control information to be 
described. When the page stream has the control information 
(Step 1409). the stream creation unit 1202 receives input of 

40 a name of the input signal (Step 1410), and receives input of 
commands to be executed according to the name of the input 
signal (Step 1411). The stream creation unit 1202 writes the 
name of the inputted input signal and the command in non 
working area of the storage area allocated to the items of the 

45 control information of the page stream in the page stream 
storage unit 1207 (Step 1412). and then goes back to Step 
1409. 

In Step 1409. when the contents to be described as the 
partial image of the control information do not exist, the 

30 stream creation unit 1202 further requests input instructing 
whether the page stream has animation display to be 
described. When the animation display exists (Step 1413). 
the stream creation unit 1202 receives input of the partial 
image ID of the partial image to be displayed in animation 

55 (Step 1414). and receives input of one or a plurality of types 
of animation instructing how the partial image of the partial 
image ID should be displayed in animation (Step 1415). The 
types of animation is chosen from "movement", "enlarge", 
"rotation", and the like. Aparameter. which changes with the 

6C passage of time is designated according to the input of the 
type of animation. The stream creation unit 1202 requests 
input instructing whether a pair of a time and a value, both 
of which should be shown by the page stream, exists; the 
time being expressed by the elapsed time from the start of 

65 the execution of the time line information and the value 
corresponding to the types of inputted animation, and the 
time mentioned above. When the pair exists (Step 1416), the 
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stream creation unit 1202 receives the input (Step 1417). 
Specifically, as parameters corresponding to the time men- 
tioned above, values of the X-Y coordinates, the X-Y 
enlargement rates, and the rotational angle are inputted for 
"movement", "enlargement**, and "rotation", respectively. 5 

In Step 1416. when the pair does not exist, the stream 
creation 1202 writes the partial image ID of the partial image 
to be displayed in animation and the pair of the time and the 
parameter in the non working are as of the storage areas 
allocated to the information relating to the animation display 1C 
of the page stream in the page stream storage unit 1207 (Step 
14 IS), and goes back to Step 1413. 

In Step 1413. when the animation display to be described 
does not exist in the page stream, the stream creation unit 
1202 requests input instructing whether dynamic picture 15 
display to be described exists in the page stream. When the 
dynamic picture display to be described exists (Step 1419). 
the stream creation unit 1202 receives input of a start and 
end times of the dynamic picture display, and the partial 
image ID of the partial image which should be displayed in 20 
dynamic picture (dynamic picture display object) (Step 
1420). 

The stream creation unit 1202 further receives input of the 
name of the dynamic picture file which shows the dynamic 
picture to be displayed inside the partial image shown by the 25 
inputted partial image ID (Step 1421). 

Next, the stream creation unit 1202 requests input 
instructing whether there is an audio file which shows sound 
to be played back synchronized with the dynamic picture 3Q 
shown by the name of the inputted file. When the audio file 
exists (Step 1422). the stream creation unit 1202 receives 
input of the name of the audio file (Step 1423). Next, the 
stream creation unit 1202 reads the dynamic picture file and 
audio file from the dynamic picture storage unit 1205 and the ^ 
audio storage unit 1206. respectively, and writes the still 
picture data and a pair of still picture data and audio data of 
the same time in the storage areas allocated to the time line 
stream having the identification number in the time line 
stream storage unit 1208 {Step 1424). ^ 

The stream creation unit 1202 writes the start and end 
times of the inputted dynamic picture display, and the partial 
image ID of the dynamic picture display object, along with 
the identification number of the time line stream, in the non 
working areas allocated to the information relating to the 45 
dynamic picture display of the page stream in the page 
stream storage unit 1207 (Step 1425). 

In Step 1422, when the audio file does not exist, the 
stream creation unit 1202 reads the dynamic picture file 
whose name is inputted from the dynamic storage unit 1205. x 
and writes the contents of the file in the storage areas 
allocated to the time line stream having the identification 
number in the time line stream storage unit 1208 (Step 
1426). and then starts Step 1425. 

Next, the stream creation unit 1202 increments the iden- 55 s 
tification number of the time line scream (Step 1427). and 
goes back to Step 1419. 

In Step 1419. when the dynamic picture display to be 
described does not exit in the page stream, the stream 
creation unit 1202 increments the identification number of 60 
the page stream (Step 1428), and goes back to Step 1402. j 

In Step 1402. when the contents to be described do not j 
exist in the page stream, the stream creation 1202 requests 
the input instructing order of recording the page scream and l 
the time line stream successively onto the storage areas of 65 I 
the track of the CD-ROM 1210 (Step 1429). The stream ! 
creation unit 1202 posts the recording order to the CD-ROM 
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write unit 1209. The CD-ROM write unit 1209 reads the 
page stream and the time line stream from the page stream 
storage unit 1207 the time Line stream storage unit 1208. 
respectively, and stores them in the CD-ROM 1210 (Step 
5 1430;. 

As mentioned above, one or a plurality of partial images 
can be shown by one page stream, and multi-media can be 
recorded in the CD-ROM 1210: the multi-media information 
in which parameters for moving, enlarging/reducing, rotat- 

ic ing partial images, commands to be executed according to 
the input signal, and the time line information are described 
by the page stream. Also, each page stream and time line 
stream can be stored successively in the storage areas on the 
same track of the CD-ROM 1210. According to the present 

15 embodiment, the multi-media information stored in the 
multi-media information record device 1200 can be played 
back from the CD-ROM 1210 effectively by the multi-media 
information playback device 200. 
According to the present embodiment, the still picture 

20 data and the audio data shown by the time line stream are 
compressed and encoded according to the MPEG, and they 
are extended and coded by the MPEG decoder 311. But. the 
still picture data, the audio data, and the partial image data 
can be compressed and encoded by other formats like VFW 

25 {Video for Windows) or QT (QuickTime). And the MPEG 
decoder 311 can be substituted for a decoder following other 
formats. When the multi-media information record device 
1200 only deals with data which is not compressed and 
encoded, there is no problem without a decoder. 

30 (Embodiment 3) 

FIG. 16 is a block diagram showing construction of the 
multi-media information playback device 1600 of the third 
embodiment of the present invention. FIG. 17 is a block 

35 diagram showing hardware construction of the multi-media 
information playback device shown in FIG. 16. 

As shown in FIG. 16. the multi-media information play- 
back device 1600 is connected to a data server 1700 through 
a network, comprising a network interface unit 1601 and a 

43 page load unit 1602. Other elements are the same as the 
multi-media information playback device 200 shown in FIG. 
2. so only the same reference numbers are given without 
explanation. The multi-media information can be equipped 
with the CD-ROM 201 and the CD-ROM interface unit 202. 

45 As shown in FIG. 17. the hardware construction of the 
multi-media information playback device 1600 is the same 
as that of the multi-media information playback device 200 
shown in FIG. 3 except that it comprises a network interface 
1701. Explanation of the hardware construction is omitted. 

50 since it is the same as that of the multi-media information 
playback device 200. so only the same reference numbers 
are given. 

The data server 1700 is an external device comprising 
storage devices like a hardware disc device, which is not 

55 illustrated The data server 1700 stores the page stream 
shown in FIG. 5 and the time line stream shown in FIG. 6 
in the storage device, one page stream being stored as one 
file and one time line stream as one file. The data server 1700 
reads files which contains the corresponding page stream 

6C and time line stream from the hard disc device according to 
the read request from the multi-media information playback 
device 1600 via the network, and transfers the page stream 
and the time line stream to the multi-media information 
playback device 1600. 

65 The network interface unit 1601 is implemented by a 
network interface 1701. According to the instruction of the 
page load unit 1602, this unit 1601 reads multi-media 
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information described by the page stream and the time line 
stream stored in the data server 1700. Specifically, the 
network interface unit 1601 transmits FT partition showing 
whether the read request information is the page stream 
f'F*) or the rime line stream CT") as for the page stream or 5 
the time line stream designated by the page load unit 1602. 
along with the identification number and the read request, to 
the data server 1700 via the network. The network interface 
unit 1601 receives the page stream or the time line stream 
transmitted from the data server 1700 according to the read 10 
request via the network. 

The page load unit 1602 is implemented by the CPU 301. 
Processing of the page load unit 1602 is the same as that of 
the page load unit 206 of the first embodiment except that 
the read request of the page stream and the time Line stream 15 
are given to the network interface unit 1601 instead of the 
CD-ROM interface unit 202. 

Explanation about the procedure of the playback of the 
multi-media inf ormation playback device 1600 follows next, 
only focusing the difference between the multi- media infor- 
mation playback devices 1600 and 200. 

In Step 701, when the multi-media information playback 
device 1600 is activated, the page load unit 1602 instructs 
the network interface unit 1601 to read a predetermined page ^ 
stream, e.g. the page stream 1 shown in HG. 8. from the data 
server 1700. The network interface unit 1601 transfers a PT 
partition **P". an identification number **r and its read 
request to the data server 1700, and receives the page stream 
"P from the data server 1700. The page load unit 1602 loads M 
the page stream to the page storage device 210. 

The following explanation is about Steps 705-707 when 
the page stream 1 is loaded. 

When the identified input unit is B. the control informa- 
tion execution unit 204 executes a command "Start the clock 35 
of the time line.** which corresponds to the input signal B. 
Specifically, the control information execution unit 204 
instructs the time line control unit 207 to start the execution 
of the time line information described by the page stream 1. 
Next, the time line control unit 207 activates the clock 205 40 
and reads the time line information in the page steam L 
When the designated time line stream exits in the time line 
information which describes the dynamic picture display, 
the rime line control unit 207 instructs the page load unit 
1602 to load the time line stream. In the page stream 1 45 
shown in FIG. 8. an identification number of the time line 
stream to be displayed in dynamic picture "1" is designated, 
so the time line control unit 207 instructs the page load unit 
1602 to load the time line stream 1 to the page storage unit 
210. The page load unit 1602 instructs the network interface 50 
unit 1601 to read the time line stream 1. and the network 
interface unit 1601 transfers a PT partition *T\ an identi- | 
fication number "i* and its read request to the data server \ 
1700. The time line stream 1 from the data server 1700 is 
loaded to the page storage unit 210. The processing after- 55 
wards is the same as that of Embodiment 1. 

When the identified input signal is C, the control infor- 
mation execution unit 204 executes a command **Read the 
page stream 2." which corresponds to the input signal C. 
Specifically, the control information execution unit 204 60 
instructs the page load unit 1602 to load the page stream 2 
to the page storage unit 210. The page load unit 1602 
instructs the network interface unit 1601 to read the page 
stream 2, The network interface unit 1601 transfers a FT 
partition "F\ an identification number "2" and its read 65 
request to the data server 1700. The page load unit 1662 
loads the page stream 2 . which is transferred to the network 



10 



20 



24 

interface unit 1601 from the data server 1700. to the page 
storage unit 210. The processing afterwards is the same as 
that of Embodiment 1. 

As mentioned above, the multi-media information play- 
back device 1600 produces the same effect as that of the 
multi-media information playback device 200 of Embodi- 
ment 1 by the page stream and the time line stream from the 
data server 1700* according to the present embodiment. 

The data server 1700 does not necessarily have to store 
the page stream and the time line stream, separately. The 
data server 1700 can store them by the identification num- 
bers of each file expressed by ordering numbers and the like. 
In this case, designation of the page stream and the time line 
stream described by the page stream information and the 
15 time line stream informadon. and the designation of the page 
stream and the time line stream by the parameters of the 
control information execution unit 204 and the time line 
control unit 207 are carried out by the identification numbers 
of each file. 

In the present invention, the contents of the time line 
stream do not necessarily have to be loaded to the page 
storage unit 210 at a time. For example, the page load unit 
206 can load the designated still picture data and the audio 
data in the designated time line stream to the page storage 
25 unit 210 at every display time of the dynamic picture 
display: the display data creation unit 213 can read the 
loaded still picture data from the page storage unit 210 and 
create display data; the time line control unit 207 can 
transfer the loaded audio data to the audio data storage unit 
30 211 . The following processing should be carried out in order 
to realize the variations mentioned above. The network 
interface unit 1601 and the data server 1700 reserve the 
communication areas of the network for a given time 
corresponding to the dynamic picture display execution 
35 time. Then the data server 1700 reads the still picture data 
and the audio data when they are compressed and encoded 
according to the MPEG and the like from the file which 
records time line stream of the read request The page load 
unit 1602 loads the still picture image data and the audio 
40 data to the page storage unit 210 in the multi-media infor- 
mation playback device 1600 which receives the still picture 
data and the audio data corresponding to each display time 
via the network- The loaded still picture data and the audio 
data are played back at the pr oper display time by each of the 
45 elements of the multi-media information playback device 
1600. In the data server 1700. one time line stream is stored 
in one file. So when the still picture data and the audio data 
in the same time line stream are read, read request to the data 
server 1700 does not happen. So, according to the present 
50 invention, even though the contents of the time line stream 
is not loaded to the page storage unit 210 at a time, the still 
picture data and audio data corresponding to the display time 
from the same time line stream are read at high speed So 
breaks does not happen in the sound and dynamic pictures. 
55 When the dynamic picture display, the time line control 
unit 207 can directly instruct the network interface unit 1601 
to read the designated time line stream. 
(Embodiment 4) 
60 FIG- 18 is a block diagram showing the construction of 
the multi-media informadon record device 1800 of the 
fourth embodiment of the present embodiment. FIG. 19 is a 
block diagram showing hardware construction of the multi- 
media informadon record device 1800 shown in FIG. 18. 
65 As shown in FIG. 18. the multi-media information record 
device 1800 is connected to the data server 1700. instead of 
the CD-ROM write unit 1209 and the CD-ROM 1210 of the 
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multi-media information record device 1200 shown in FIG. 
12. via a communication line, and comprises a data server 
write unit 1S01. Other elements are the same as the multi- 
media information record device 1200. so explanation is 
omitted. Only reference numbers are given. 5 

As shown in FIG. 19, the hardware construction of the 
multi -media information record device 1800 comprises a net 
interface 1901. instead of the CD-ROM 1309 of the multi- 
media information record device 1200 shown in FIG. 13. 
Other elements axe the same as the multi-media information 10 
record device 1200 shown in FIG. 13. So explanation is 
omirted. Only reference numbers are given. 

The data server 1700 receives the page stream and the 
time line stream transferred from the data server write unit 
1801 along with a write request via a communication line. 15 
and stores them in the storage devices inside* e.g. a hard disc 
device which is not illustrated, as individual file. 

The data server write unit 1801 is implemented by the 
network interface 1901. The processing of the data server JQ 
write unit 1801 is the same as that of the CD-ROM write unit 
1209 shown in FIG. 12, but the created page stream and the 
time line stream are stored in the storage device inside the 
data server 1700 which is connected via a communication 
line, instead of the CD-ROM 1210. The data server write 2$ 
unit 1801 reads the time line stream from the time line 
stream storage unit 1208 according to the notification from 
the stream creation unit 1202, and transfers a FT partition 
*T' of the time line stream, an identificatioa number* and the 
time line stream to the data server 1700 along with the write ^ 
request. Then the data server 1700 records the page stream 
in the file having the PT partition "P M and the identification 
number, and records the time line stream in the file having 
the FT partidon *T* and the identification number. 

According to the present embodiment, the multi-media 33 
information record device 1800 can stored the page stream 
and the time line stream shown in FIG. 5 and HG. 6 in the 
storage device inside the data server 1700 as one file. The 
present embodiment can produce the same effect mentioned 
in Embodiment 1 by playing back the page stream and the ^ 
time line stream recorded, by the multi-media information 
record device 1800. in the multi-media information play- 
back device 1600 connected to the data server 1700 via a 
network. The data server 1700 can store the page stream and 
the time line stream by the identification numbers of each 45 
file, without distinguishing the page stream and the time line 
stream. In this case, the designation of the page stream and 
the time line stream described in the information in the page 
stream and the time line stream, and the designation of the 
page stream and the time line stream by the parameters of ^ 
the control information execution unit 204 and the time line 
control unit 207, are carried out by the identification num- 
bers of each of the files. 

Although the present invention has been fully described 
by way of examples with reference to the accompanying 55 
drawings, it is to be noted that various changes and modi- 
fications will be apparent to those skilled in the art. 
Therefore, unless such changes and modifications depart 
form the scope of the present invention, they should be 
construed as being included therein. ec 
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What is claimed is: 

1 . A multi-media infonnation playback device which plays back multi- 
media information [recorded in a recording medium] comprising: 

(a) a [disc] recording medium having pieces of page information, 
each of which is made up of multi-media information that describes contents 

5 of a plurality of screen images a [successively] stored in a storage area, the 

page information including; 

(1) a partial image list having a pair of partial image 
information showing a partial image of a screen image and display 
1 state information showing display state of the partial image, and 

1 0 (2) control information which describes a command to be 

executed according to an input signal, including a display state 
change command instructing to change the display state of the partial 
image; 

(b) a page information read means for reading a piece of page 
1 5 information from the [disc] recording medium : 

(c) a display data creation means for creating display data of the 
screen image by placing the partial image shown by the partial image 
information on a screen; 

(d) a display means for displaying each display data created by 
20 ■ the display data creation means; 
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(e) an input means for receiving an input signal from an 
operator; and 

(f) a command execution means for detecting the input signal 
inputted by the operator, judging whether the input signal corresponds to the 
command described by the control information in the page information read 
by the page information read means, and instructing the display data creation 
means to create new display data by changing the display state of the 
corresponding partial image according to the display state change command 
when the input signal corresponds to the display state change command; 

wherein the display data creation means creates new display data 
according to the command execution means. 



2. The multt-media information playback device of claim 
1. wherein the page information includes: 

(a) the partial image list; 

(b) the control information including an animation display 
command instructing to change and display the display 
state information of the partial image corresponding to 

^ elapsed time; and 

(c) time sequence control information including anima- 
tion control information which describes %'alues of the 
display state information of the partial image corre- 
sponding to the elapsed time; 

45 wherein the command execution means includes: 

(a? a first timer for measuring the elapsed time after 

start of activation; and 
(b) an animation display command execution unit for 
activating the first timer when an input signal cor- 
50 responding to the animation display command is 

detected, calculating values of the display state infor- 
mation corresponding to the elapsed time of each of 
the partial images based on the animation control 
information at predetermined times, instructing the 
55 display data creation means to create new display 

data by changing the display state of the panial 
image according to the calculated values of the 
display state information, and stopping the first timer 
when execution of the animation display command 
6C is completed; 

wherein the display data creation means creates new 
display data according to the animation display com- 
mand execution unit 

3. The multi-media information playback device of claim 
65 2. wherein the page information includes: 

(a; the partial image list having a pair of the partial image 
information and display state information, the display 
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state information including X-Y coordinates showing a 
display position of the partial image to be displayed on 
jht screen. X-Y enlargement rates showing a ratio of 
size of the partial image shown by the partial image 
informadon and a partial image to be displayed on the 5 
screen, and a rotational angle between the partial image 
shown by the partial image information and a partial 
image to be displayed on the screen: and 
Cb) time sequence control informadon including the ani- 
mation control information which describes at least one *s 
among the values of the X-Y coordinates, the X-Y 
. enlargement rates, and the rotational angle correspond- 
ing to the elapsed time; 
wherein the animation display command execution unit 
includes: 15 
(a; an X-Y coordinates calculation unit for calculating 
the X-Y coordinates corresponding to the elapsed 
time based on the animation control information at 
predetermined times; 

(b) an X-Y enlargement rate calculation unit for cal- 20 
cuhting the X-Y enlargement rates corresponding to 
the elapsed time based on the animation control 
information at predetermined times; and 

(c) a rotational angle calculation unit for calculating the 
rotational angle corresponding to the elapsed time 25 
based on the animation control information at pre- 
determined times. 



4. The multi-media information playback device of Claim 3, 
wherein there is at least one storage area on the [disc] recording medium for 
[successively] storing page information and at least one storage area for 
[successively] storing time sequence information, the page information 
5 including: 

(a) the partial image list, 

(b) the control information including a time sequence 
processing start command instructing to start execution of time 
sequence processing corresponding to the elapsed time, and 

0 (c) time sequence control information including time 

sequence processing control information having a name of time 
sequence information to be played back according to the time 
sequence processing start command, IDs showing contents of the 




time sequence information to be played back at predetermined 
times, and playback state information describing how the time 
sequence information should be played back in accordance with 
display of the partial image shown by the partial image 
information in the partial image list; 

wherein the time sequence information has information, 
which should be played back according to the elapsed time, 
stored in order of playback; 

wherein the command execution means includes: 

(a) a second clock unit for measuring the elapsed 
time after the start of activation; and 

(b) a time sequence processing execution unit for 
activating the second clock means when an input signal 
corresponding to the time sequence processing start command is 
detected, designating contents of the time sequence information 
corresponding to the elapsed time, and stopping the second clock 
unit when playback of the time sequence information is 
completed; 

wherein the multi-media information playback device 
further includes: 

(a) a page storage means, which is implemented by 
RAM, for storing the page information read by the page 
information read means; 




(b) a time sequence information read means for 
reading the time sequence information designated by the time 
sequence processing execution unit from the [disk] recording 
medium ; and 

(c) a playback means for playing back the time 
sequence information read by the time sequence information read 
means, and instructing the display means to display the time 
sequence information. 

5. The multi-media information playback device of claim 
jc 4. wherein the page information includes: 

(a) the partial image list. 

(b) the control information, and 

{c; time sequence control information including time 

^ sequence processing control information having a 
name of first time sequence information as time 
sequence information to be played back according to 
the time sequence processing start command, IDs of 
sound information to be played back at predetermined 
times* and playback state information which describes 
how sound information should be played back in accor- 
dance with display of the partial image shown by the 
partial image information in the partial image list: 
wherein the time sequence information includes the first 

25 time sequence information comprising sound informa- 
tion corresponding to the ID of sound information, the 
sound information being divided into equal predeter- 
mined sizes and placed in order of playback corre- 
sponding to the elapsed time: 

3C wherein the time sequence processing execution unit 
includes a first time sequence processing execution unit 
for designating the sound information in the first time 
sequence information corresponding to the elapsed 
time at predetermined times, when a name of the first 

35 time sequence information is shown by the time 
sequence processing control information: 
wherein the playback means includes a sound playback 
unit for playing back the sound information. 



31 



6. The multi-media inf ormation playback device of claim 
40 4* wherein the page information includes: 
(a) the partial image list, 
(b> the control information, and 

(c) the time sequence control information including a 
45 name of a second time sequence information as time 
sequence information to be played back according to 
the time sequence process ing start command, pairs of 
IDs of sound information and image information to be 
played back at predetermined times, and the playback 
state information which describes at least one of 
dynamic picture and animation should be displayed in 
an area relating to the partial image shown by the 
partial image information; 
wherein the time sequence information includes the sec- 
55 ond time sequence information comprising sound 
information divided into equal predetermined sizes and 
image information showing an image of at least one of 
the dynamic picture and the animation, the sound 
information and the image information being placed in 
60 order of playback corresponding to the elapsed time; 
wherein the time sequence processing execution unit 
includes a second time sequence processing execution 
unit for designating sound information and image infor- 
mation in the second time sequence information cor- 
es responding to the elapsed time at predetermined times 
when the second time sequence information is shown 
by the time sequence processing control information, 
and instructing the display data creation means to 
create new display data by placing the partial image 
read by the time sequence information read means in an 
area shown by the playback state information; 
wherein the display data creation means creates new 
display data according to the second time sequence 
processing execution unit. 



7. (Amended) A multi-media information record device 
for recording multi-media information in a disc, comprising: 

(a) a partial image storage means for storing the 
partial image information of at least one partial image in one 
partial image file, the partial image information being a part 

of the screen image; 

(b) a partial image list input means for receiving 

input of a name of at least one partial image file in the partial 
image storage means and display state information showing 
display state of the partial image shown by the partial image 
information in the partial image file; 




(c) a page information storage means for having 
a plurality of separate small storage areas, each of which is 
used for storing at least a piece of page information which is 
made up of multi-media information that describes contents 
of a plurality of screen images; 

(d) a partial image list write means for writing a 
pair of the partial image information and the display state 
information in a first small storage area in the page 
information storage means along with the Ids of the partial 
image information and the display state information, after 

reading partial image information from the partial image file 
whose name is inputted, the display state information 
being inputted according to the partial image 
information; 

(e) a control information input means for 
receiving input of control information which includes a name 
of input signal and display state change command instructing 
to change display state of the partial image corresponding to 
the input signal; 

(f) a control information write means for writing 
the inputted control information in a second small storage 
area in the page information storage means; and 
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(g) a P a S e information record means for 
[successively] recording information, which is written as a 
piece of page information in each small storage area in the 
page information storage means, in a storage area on the disc. 



8. The multi-media information record device of claim 7. 
wherein the control information deludes animation display 
command instructing to change and display state informa- ^ 
don of the partial image corresponding to the elapsed time: 
wherein the control information input means further 
* includes an animation display command input unit for 

receiving input of a pair of an input signal and the 
animation display command: 55 
wherein the multi-media information record device fur- 
ther includes: 

(a) a time sequence control information input means for 
receiviag time sequence control information includ- 
ing animation control information, the animation 6C 
control information describing how display state of 
the partial image changes according to the elapsed 
time; and 

(b) a time sequence control information write means for 
writing the inputted time sequence control informa- 65 
tion Ln a third small storage area in the page infor- 
mation storage means. 

9. The multi-media information record device of claim 8. 
wherein the partial image list input means further includes^ 
display state information input unit for receiving input of 
display state information for each piece of partial image 
5 information, the display information including X-Y coordi- 
nates showing a display position of the partial image to be 
displayed on the screen. X-Y enlargement rates showing a 
ratio of size of the partial image shown by the partial image 
information and a partial image to be displayed on the 
to screen, and a rotational angle between the partial image 
shown by the partial image information and a partial image 
to be displayed on the screen: 
wherein the time sequence control information input 
means receives input of animation control information 
15 which describes how at least one among the values of 
the X-Y coordinates, the X-Y enlargement rates, and 
the rotational angle changes corresponding Co the 
elapsed time. 
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10. A multi-media information playback device for read- 
ing multi-media information from a data server connected 
via a network and playing back the multi-media information, 
comprising: 

(a; a page information read means for reading a piece of 
page information, which is made up of multi-media 
information that describes concents of a plurality of 
screen images, stored as one rUe from a data server 
connected via a network, the page information includ- 
ing: 

(ij a partial image list having at least one pair of the 
partial image information showing a partial image of 
a screen image and display state information describ- 
ing display state of the partial image, and 

(2) control information which describes a command to 
be executed according to an input signal, including a 
display state change command instructing to change 
the display state of the partial image; 
fb) a display data creation means for creating display data 

of the screen image by placing (he partial image shown 

by the partial image information on a screen; 

(c) a display means for displaying each display data 
creared by the display data creation means; 

(d) an input means for receiving an input signal from an 
operator; and 

(e> a command execution means for detecting the input 
signal inputted by the operator, judging whether the 
input signal corresponds to the command described by 
the control information in the page information read by 
the page information read means, and instructing the 
display data creation means to create new display data 
by changing the display state of the corresponding 
partial image according to the display state change 
command when the input signal corresponds to the 
display state change command; 

wherein the display data creation means creates new 
display data according to the command execution 
means. 
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11. The multi- media information playback device of 
claim 10* wherein the page information including: 
fa) the partial image list; 

(b) the control information including the animation dis- 
play instructing to change and display stare information 
of the partial image corresponding to the elapsed time; 

65 and 

(c) time sequence control information including anima- 
tion control information which describes values of 

display state information of the partial image corre- 
sponding to the elapsed time; 
wherein the command execution means includes: 

(a) a first timer for measuring the elapsed time after 
start of activation; and 3 

(b) an animation display command execution unit for 
activating the first timer when an input signal cor- 
responding to the animation display command is 
detected, calculating values of the display state infor- 
mation corresponding to the elapsed time of each of *° 
the partial images based on the animation control 
information at predetermined times, in strutting the 
display data creation means to create new display 
data by changing the display state of the partial 
image according to the calculated values of the l $ 
display state information, and stopping the first timer 
when execution of the animation display command 

is completed; 

wherein the display data creation means creates new 
display data according to the animation display com- 20 
mand execution unit 
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12. The multi-media information playback device of 
claim 11. wherein the page information including: 

(a) the partial image list having of at least one pair of the 
partial image information and the display stare infor- 25 
mation including X-Y coordinates showing a display 
position of the partial image to be displayed on the 
screen, X-Y enlargement rates showing a rado of size 
of the partial image shown by the partial image infor- 
mation and the partial image to be displayed on the J0 
screea. and a rotational angle between the partial image 
shown by the partial image information and the partial 
image to be displayed on the screen; 

(b) the control informadon including the animation dis- 
play command; and 35 

(c) time sequence control information including the ani- 
mation control information which describes at least one 
among the values of the X-Y coordinates, the X-Y 
enlargement rates, and the rotational angle correspond- ^ 
ing to the elapsed time; 

wherein the animation display command execution unit 
includes: 

(a) an X-Y coordinates calculation unit for calculating 
the X-Y coordinates corresponding to the elapsed 45 
time based on the animation cootrol information at 
predetermined times; 

{b} an X-Y enlargement rate calculation unit for cal- 
culating the X-Y enlargement rates corresponding to 
the elapsed time based on the animation control 5Q 
information at predetermined times; and 

(c) a rotational angle calculation unit for calculating the 
rotational angle corresponding to the elapsed time 
based on the animation control information at pre- 
determined times. 5S 

13. The } multi-media information playback device of 
claim 12. therein the page information including: 

(a) the partial image list; 

(b) the control information Including a time sequence 
processing start command instructing to start execution 6G 
of time sequence processing corresponding to the 
elapsed time; and 

(c; time sequence control information including time 
sequence processing control information having a 
name of time sequence information to be played back 65 
according to the time sequence processing start 
comma ad. Ids showing contents of the time sequence 
information to be played back at predetermined times, 
and playback state information describing how the time 
sequence information should be played back in accor- 
dance with display of the partial image shown by the 
5 partial image information in the partial image list; 
wherein the command execution means includes: 
(a> a second timer for measuring the elapsed time after 

the start of activation; and 
(bj a time sequence processing execution unit for 
10 activating the second timer when an input signal 

corresponding to the time sequence processing staxt 
command is detected, designating contents of the 
time sequence information corresponding to the 
elapsed time, and stopping the second timer when 
15 playback of the time sequence information is com- 

pleted; 

wherein the multi-media information playback device 
further includes: 

(a) a page storage means, which is implemented by 
20 RAM, for storing the page information read by the 

page information read means: 
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fb) a time sequence information read means for reading 
contents of the time sequence information desig- 
nated by the time sequence processing execution unit 
25 from the time sequence information stored as one file 

in the data server via a network, the time sequence 
information having contents in order of playback 
corresponding to the elapsed time and its name being 
included in the time sequence control information: 

(c) a playback means for playing back the time 
sequence information ready by the time sequence 
information read means, and instructing the display 
I means to display the time sequence information. 

35 14. The multi-media information playback device in 
claim 13. wherein the page information including: 

(a) the partial image list; 

(b) the control information: and 

(c) time Sequence control information including time 
40 sequence processing control information having a 

name of the first time sequence information as rime 
sequence information to be played back according to 
the time sequence processing start command, sound 
information to be played back according to the elapsed 

45 time of the first time sequence information, and 

playback state information describing how the sound 
information should be played back in accordance with 
display of the partial image shown by partial image 

^ information in the partial image list; 

wherein the time sequence processing execution unit ■ 
includes a first time sequence processing execution unit 
for designating the sound information in the first time 
sequence information corresponding to the elapsed 

55 time at predetermined times* when a name of the first 
time sequence information is shown by the time 
sequence processing control information: 
wherein the time sequence information read means reads 
the sound information designated by first time sequence 

6c processing execution unit from the first time sequence 
information stored as one file in the data server, the first 
time sequence information describing the divided 
sound information in order of playback corresponding 
to the elapsed time and its name being included in the 

$5 time sequence control information: 

wherein the playback means includes a sound playback 
unit for playing back the sound information. 

15. The multi-media information playback device of 
claim 13. wherein the page information including: 

(a) the partial image list; 

(b) the control information: and 

(c; the time sequence control information including a 
name of the second time sequence information as time 
sequence information to be played back according to 
the time sequence processing start command, a pair of 
sound information and image information to be played 
back corresponding to the elapsed time of the second 
time sequence information, and playback state infor- 
mation which describes at least one of dynamic picture 
and animation should be displayed in the area relating 
to the partial image shown by the partial image infor- 
mation; 
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wherein the time sequence processing execution unit 
includes a second time sequence processing execution 
unit for designadng sound information and image infor- 
mation in the second time sequence information cor- 
responding to the elapsed time at predetermined times 20 
when the second time sequence information is shown 
by the time sequence processing control information, 
and instructing the display data creadon means to 
create new display data by placing the partial image 
read by the time sequence information read means in an 25 
area shown by the playback state* information: 
wherein the time sequence information read means reads 
the sound information and the screen image Informa- 
tion designated by the second time sequcoce processing 
execution unit from the second time sequence infor- 
mation stored as one file in the data server, the time 
sequence information describing the divided sound 
Informadon in order of playback and the screen image 
informadon describing one screen image of at least one 
of motion picture and animation corresponding to the 
elapsed time and its name being included in the time 
sequence controL informadon; 
wherein the display data creadon means creates new 
display data according to the second time sequence ^ 
processing execution unit 
16. A muld-media information record device for record- 
ing multi-media information in a data server connected via 
a network, comprising: 

(a) a partial image storage means for storing the partial 4J 
image informadon of at least one pardai image in one 
partial image rile, the pardai image informadon being 

a part of the screen image; 

(b) a partial image list input means for receiving input of 

a name of at least one partial image fUe in the partial 50 
image storage means and display state informadon 
showing display state of the partial image shown by the 
pardai image informadon in the partial image file; 

(c) a page informadon storage means for having a plu- 
rality of separate small storage areas, each of which is 55 
used for storing at least a piece of page informadon 
which is made up of muld-media informadon that 
describes contents of a plurality of screen Images; 

(d) a partial image list write means for writing a pair of the 
partial image information and the display state infor- 60 
matlon in a first small storage area in the page infor- 
mation storage means along with the Ids of the partial 
image informadon and the display state informadon. 
after reading partial image informadon from the partial 
image file whose name is inputted, the display state 65 
informadon being inputted according to the partial 
image informadon: 

(e) a control information input means for receiving input 
of control informadon which includes a name of input 
signal and display state change command instrucdng to 
change display state of the partial image corresponding 
to the input signal: 

(f> a control information write means for writing the 
inputted control informadon in a second small storage 
area in the page information storage means: and 

(g) a page informadon record means for recording con- 
tents of a piece of page information in one file in the 
data server after transmitdng contents written as a piece 
of page informadon in each storage area in the page 
informadon storage means to a data server along with 
its write request. 
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15 17. Tne multi-media information record device of claim 

16. wherein the control information includes animation 
display command instructing to change and display state 
information of the partial image corresponding to the 
elapsed time; 

wherein the control information input means further 
includes an animation display command input unit for 
receiving input of a pair of an input signal and the 
animation display command; 
wherein the multi-media information record device fur- 
ther includes: 

(a) a time sequence control information input means for 
receiving time sequence control information includ- 
ing animation control information, the animation 
control information describing how display state of 
the partial image changes according to the elapsed 
time: and 

(bj a time sequence control information write means for 
writing the inputted time sequence control informa- 
35 tion in a third small storage area in the page infor- 

mation storage means. 

18. The multi-media information record device of claim 

17. wherein the partial image list input means further 
includes a display state information input unit for receiving 

40 input of display state information for each piece of partial 
image information, the display information including X-Y 
coordinates showing a display position of the partial image 
to be displayed on the screen. X-Y enlargement rates show- 
ing a ratio of size of the partial image shown by the partial 
45 image information and a partial image to be displayed on the 
screen, and a rotational angle between the partial image 
shown by the partial image information and a partial image 
to be displayed on the screen; 
wherein the time sequence control information input 
J0 means receives input of animation control information 
which describes how at least one among the values of 
the X-Y coordinates, the X-Y enlargement rates, and 
the rotational angle changes corresponding to the 
elapsed time. 

19. A multi-media information record device for record- 
ing multi-media information in a disc, comprising: 

(a) a partial image storage means for storing the partial 
image information of at least one partial image in one 
partial image file, the partial image information being 

&j a part of the screen image; 

(b) a partial image list input means for receiving input of 
a name of at least one partial image file in the partial 
image storage means and display state Lnformation 
showing display state of the partial image shown by the 

65 partial image information in the partial image file; 
wherein the partial image list input means further inciudes 
a display state information input unit for receiving 
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input of display state information for each piece of 
partial image information, the display information 
including X-Y coordinates showing a display position 
of the partial image to be displayed on the screen. X-Y 
enlargement rates showing a ratio of size of the partial 5 
image shown by the partial image information and a 
partial image to be displayed on the screen, and a 
rotational angle between the partial image shown by the 
partial image information and a partial image to be 
displayed on the screen; to 

(c) a page information storage means for having a plu- 
rality of separate small storage areas, each of which is 
used for storing at least a piece of page information: 

(d) a partial image list write means for writing a pair of the 
partial Image information and the display state infer- i5 
mation in a first small storage area in the page infor- 
mation storage means along with the Ids of the partial 
image information and the display state information, 
after reading partial image information from the partial 
image file whose name is inputted, the display state 20 
information being inputted according to the partial- 
image informadon; 

(ej a control information input means for receiving input 
of control information which includes a name of input ^ 
signal and display state change command instructing to 
change display state of the partiai image corresponding 
to the input signal; 

wherein the control information includes animation dis- 
play command instructing to change and display state 3Q 
information of the partiai image corresponding to the 
elapsed time; 

wherein the control information input means further 
includes as animation display command input unit for 
receiving input of a pair of an input signal and the 35 
animation .display command; 

{f; a control information write means for writing the 
inputted control information in a second small storage 
area in the page information storage means; 

(g) a page information record means for [successively] 
recording information, which is written as a piece of 
page information in each small storage area in the 
page information storage means, in a storage area on 
the disc; 

(h) a time sequence control information input means for 45 
receiving time sequence control information including 
animation control information, the animation control 
information describing how display state of the partial 
image changes according to the elapsed time; 

wherein the time sequence control information input 
means receives input of animation control information 
which describes how at least one among the values of 
the X-Y coordinates, the X-Y enlargement rates, and 
the rotational angie changes corresponding to the J5 
elapsed time; 

(i; a time sequence control information write means for 
writing the inputted time sequence control information 
in a third 'small storage area in the page information 
storage means; 
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(j) a time sequence information storage means having a 
piece of time sequence information stored in each time 
sequence information file, the time sequence informa- 
tion having information written in order of playback 
corresponding to the elapsed time: and 65 

(k) a time sequence information means for [successively] 
recording time sequence information in a storage area 
on the disc after receiving input of a name of at least 
one time sequence information file in the time 
sequence information storage means, reading time 
sequence information from the time sequence 
information file whose name is inputted and 
correlating the time sequence information with a 
name of time sequence information corresponding to 
the name of the file; 

wherein the control information input means further 

10 includes a time sequence processing start command 
input unit for receiving input of a pair of an input signal 
and a time sequence processing start command in the 
control information, the time sequence processing start 
command instructing to start execution of time 

l5 sequence processing corresponding to the elapsed time 
and being included in the control information: 
wherein the time sequence control information input 
means further includes a time sequence processing 
control information input unit for receiving input of the 

20 time sequence processing control information which 
includes a name of time sequence information to be 
played back according to the rime sequence processing 
stan command. Ids showing contents of the time 
sequence information to be played back at predeter- 

25 mined times, and playback state information which 
describes how the time sequence information should be 
played back in accordance with display of the partial 
image list* the time sequence processing control infor- 
mation being included in the rime sequence control 

30 information. 

20* The multi-media information record device of claim 
19. wherein the time sequence information includes first 
time sequence information in which sound information is 
divided into predetermined equal data sizes, the sound 
35 information being written along with its ID in order of 
playback corresponding to the elapsed time; 
wherein the time sequence- information record means 
includes a first time sequence information record unit 
for recording first time sequence information in the disc 
40 after reading the first time sequence information from 
the time sequeoce information file, whose name is 
inputted, when a name of the first time sequence 
information is inputted: and 




wherein the time sequence processing control information 
45 input unit includes first time sequence processing con- 
trol information input unit for receiving input of first 
time sequence processing control information which 
includes a name of first time sequence information to 
be played back according to time sequence processing 
50 start command of the page information. Ids of sound 
information to be piayed back at predetermined times, 
and playback state information -which describes how 
the sound information should be played back in accor- 
dance with display of the partial image shown by the 
55 partial image information in the partial image list, the 
first time sequence control information being included 
in the time sequence processing control information. 

21. The multi -media information record device of claim 
19, wherein the time sequence information record means 

60 includes a second time sequence information record unit for 
recording second time sequence information in the disc after 
reading the second time sequence information from the time 
sequence informadon file, whose name is inputted, when a 
name of the second time sequence information is inputted, 

65 the second time sequence informadon having sound infor- 
mation divided into predetermined equal data sizes, image 
information showing one screen image of at least one of 

animation and dynamic picture and Ids of the sound infor- 
mation and the image information written in order of play- 
back corresponding to the elapsed time: and 
wherein the time sequence processing control information 
input unit includes a second time sequence processing 5 
control information input unit for receiving input of 
second time sequence processing control information 
which includes a name of second time sequence infor- 
mation to be played back according to time sequence 
processing start command of the page information. 1C 
pairs of Ids of sound information and image informa- 
tion to be played back at predetermined times, and the 
playback state information describing at least one of 
dynamic picture and animation should be displayed in 
an area relating to partial image shown by the partial 
image information in the partial image list. 13 

22, A multi-media information record device for record- 
ing multi-media information in a data server connected via 
a network, comprising: 

(a) a partial image storage means for storing the partial 
image information of at least one partial image in one 20 
partial image file, the partial image information being 

a part of the screen Image: 

(b) a partial image list input means for receiving input of 
a name of at least one partial image file in the partial 
image storage means and display state information 25 
showing display state of the partial image shown by the 
partial image information in the partial image file: 

(c) a page information storage means for having a plu- 
rality of separate small storage areas, each of which is 
used for storing at least a piece of page information 30 
which is made up of multi-media information that 
describes contents of a plurality of screen images: 

(d; a partial Image list write means for writing a pair of the 
partial image information and the display state infor- 
mation in a first small storage area In the page infor- 
mation storage means along with the Ids of the partial 
image information and the display state information, 
after reading partial image information from the partial 
image file whose name is inputted, the display state ^ 
information being inputted according to the partial 
image information: 
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wherein the partial image list input means further includes 
a display state information input unit for receiving 
input of display state information for each piece of 45 
partial image information, the display information 
including X-Y coordinates showing a display position 
of the partial image to be displayed on the screen. X-Y 
enlargement rates showing a ratio of size of the partial 
image shown by the partial image information and a ^ 
partial image to be displayed on the screen, and a 
rotational angle between the partial image shown by the 
partial image information and a partial image to be 
displayed on the screen; 

(e) a control information input means for receiving input 55 
of control information which includes a name of input 
signal and display state change command instructing to 
change display state of the partial image corresponding 
to the input signal: 

wherein the control information includes animation dis- eo 
play command instructing to change and display state 
information of the partial image corresponding to the 
elapsed time; 

wherein the control information input means further 
includes an animation display command input unit for 65 
receiving input of a pair of an input signal and the 
animation display command; 

(0 a control information write means for writing the 
inputted control information in a second small storage 
area in the page information storage means; 
(g) a page information record means for recording con- 

5 tents of a piece of page information in one file in the 
data server after transmitting contents written as a piece 
of page information in each storage area in the page 
information storage means to a data server along with 
its write request; 

1C (h) a time sequence control information input means for 
receiving time sequence control information including 
animation control information, the animation control 
information describing how display state of the partial 
image changes according to the elapsed time; 

15 wherein the time sequence control information input 
means receives input of animation control information 
which describes how at least one among the values of 
the X-Y coordinates, the X-Y enlargement rates, and 
the rotational angle changes corresponding to the 

20 elapsed time; 

(i) a time sequence control information write means for 
writing the inputted time sequence control information 
in a third small storage area in the page information 

^ storage means; 

(j) a time sequence information storage means having a 
piece of time sequence information stored in each time 
sequence information file, the time sequence informa- 
tion having information written in order of playback 

30 corresponding to the elapsed time; and 

(k) a time sequence information record means for record- 
ing contents of time sequence information in one file in 
the data server for each piece of time sequence 
information, after receiving a name of at least one time 

35 sequence information file in the time sequence infor- 
mation storage means, reading time sequence informa- 
tion from the time sequence information file whose 
name is inputted, correlating the time sequence infor- 
mation with a name of time sequence information 

40 corresponding to the file, and transmitting the time 
sequence information to the data server along with its 
write request; 
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wherein the control information input means further 
includes a time sequeoce processing start command 
45 input unit for receiving input of a pair of an input signal 
and a time sequence processing start command in the 
control information, the time sequence processing stait 
command instructing to start execution of time 
sequence processing corresponding to the elapsed time 
sc and being included in the control information: wherein 
the time sequence control information input means 
further includes a time sequence processing control 
information input unit for receiving input of the time 
sequence processing control information which 
55 includes a name of time sequence information to be 
played back according to the time sequence processing 
start command. IDs showing contents of the time 
sequence information to be played back at predeter- 
mined times, and playback state information which 
60 describes how the time sequence information should be 
played back in accordance with display of the partial 
Image shown by the partial image information in the 
partial image list, the time sequence processing control 
information being included in the time sequence con- 
65 trol information. 

23* The multi-media information record device of claim 
22, wherein the time sequence information includes first 
time sequence information in which sound information is 
divided into predetermined equal data sizes, the sound 
information being written along with its ID in order of 
playback corresponding to the elapsed time; wherein the 
time sequence information record means includes a first time 5 
sequence information record unit for recording first time 
sequence information in the disc after reading the first time 
sequence information from the time sequence information 
file, whose name is inputted, when a name of the first time 
sequence information is inputted; wherein the time sequence 
processing control information input unit includes first time 
sequence processing control information Input unit for 
receiving input of first time sequence processing control 
information which includes a name of first time sequence 
information to be played back according to time sequence 
processing start command of the page information* IDs of 15 
sou nd information to be played back at predetermined times, 
and playback state information which describes how the 
sound information should be played back in accordance with 
display of the partial image shown by the partial image 
informatioa in the partial image list, the first time sequence 2a 
control information being included in the time sequence 
processing control information* 

24. The multi-media information record device of claim 
22. wherein the time sequence information record means 
includes a secoad rime sequence information record unit for 25 
recording second time sequence information in the disc after 
reading the second time sequence information from the time 
sequence informatioa file, whose name is inputted, when a 
name of the second time sequence information is inputted, 
the second time sequence information having sound infor- 
mation divided into predetermined equal data sizes, image 30 
information showing one screen image of at least one of 
* animation and dynamic picture and IDs of the sound infor- 
mation and the image Information written in order of play- 
back corresponding to the elapsed time; 
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wherein the time sequence processing control information - - 
input unit includes a second time sequence processing 
control information input unit for receiving input of 
second time sequence processing control information 
which includes a name of second time sequence infor- 
mation to be played back according to time sequence 4C 
processing start command of the page information, 
pairs of Ins of sound information and image informa- 
tion to be played back at predetermined times, and the 
playback state information describing at least one of 
dynamic picture and animation should be displayed in 45 
an area relating to partial image shown by the partial 
image information in the partial image list. 
25. A multi-media information playback device which 
plays back multi-media information recorded in a recording 
medium comprising: so 

a disc having pieces of page information [successively] 
stored in a storage area, each piece of page information 
being made up of multi-media information that describes 
contents of a plurality of screen images, the page 
information including: 

(1) a partial image list having a pair of partial image 
information showing a partial image of a screen 
image and display state information showing display 
state of the partial image, 

(2) control information which describes a command to 60 
be executed according to an input signal, the control 
information including: 

a display state change command instructing to 

change the display state of the partial image; 
an animation display command Instructing to change 65 
and display the display state information of the 
partial image corresponding to elapsed time: and 
a time sequence processing start command instruct- 
ing to start execution of time sequence processing 
corresponding to the elapsed time: 
(3; time sequence control information having 
5 information, which should be played back according 

to the elapsed time, stored in order of playback, the 
time sequence control information including: 
{a) animation control information which describes 
values of the display state information of the 
ic partial image corresponding to the elapsed time; 

and 

(b) time sequence processing control information 
having a name of time sequence information to be 
played back according to the time sequence pro- 
15 cesstng start command. IDs showing contents of 

the time sequence information to be played back at 
predetermined times, and playback state informa- 
tion describing how the time sequence informa- 
tion should be played back in accordance with 
20 display of the partial image shown by the partial 

image information in the partial image list; 
(b) a page information read means for reading a piece of 

page information from the disc; 
(c> a display data creation means for creating display data 
25 of the screen image by placing the partial image shown 
by the partial image information on a screen; 
(d) a display meaas for displaying each display data 
created by the display data creation means; 




30 (e) an input means for receiving an input signal from an 
operator: and 

(f) a command execution means for detecting the input 
signal inputted by the operator, judging whether the 
input signal corresponds to the command described by 

35 the control information in the page information read by 
the page information read means, and instructing the 
display data creation means to create new display data 
by changing the display state of the corresponding 
partial image according to the display state change 

4C command when the input signal corresponds to the 
display state change command; the command execu- 
tion means including: 

(1) a first timer for measuring the elapsed time after 
start of activation: 

45 (2) an animation display command execution unit for 
activating the first timer when an input signal cor- 
responding to the animation display command is 
detected, calculating values of the display state inf or- 
mation corresponding to the elapsed time of each of 

50 the partial images based on the animation control 

information at predetermined times, instructing the 
display data creation means to create new display 
data by changing the display state of the partial 
image according to the calculated values of the 

55 display state information, and stopping the first timer 

when execution of the animation display command 
is completed; 

(3; a second timer for measuring the elapsed time after 
the start of activation; and 

60 (4) a time sequence processing execution unit for 
activating the second timer when an input signal 
corresponding to the time sequence processing start 
command is detected, designating contents of the 
time sequence information corresponding to the 

65 * elapsed time, and stopping the second timer when 
playback of the time sequence information is com- 
pleted; 

wherein the display data creation means creates new 
display data according to the animation display com- 
mand execution unit; 
f (g) a page storage means, which is implemented by RAM. 

for Storing the page information read by the page 5 
information read means; 

Ch) a time sequence information read means for reading 
the time sequence information designated by the time 
sequence processing execution unit from the disk; and 

(i> a playback means for playing back the time sequence 10 
information read by the time sequence information read 
means, and instructing the display means to display the 
time sequence information. 

26. A multi-media informadon playback device which 
plays back multi-media information recorded in a recording 
medium comprising: 

(a) a disc having pieces of page information [successively] 
stored as one file from a data server connected via a 
network, each piece of page information being made up of 
multi-media information that describes contents of a 



plurality of screen images, the page information including: 
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{ 1; a partial image list having a pair of partial image 
information showing a partial image of a screen 25 
image and display state information showing display 
state of the partial image. 

(2) control information which describes a command to 
be executed according to an input signal, the control 
information including: 3C 
a display state change command instructing to 

change the display state of the partial image; 

an animation display command instructing to change 
and display the display state information of the 
partial image corresponding to elapsed time; and 35 

a time sequence processing start command instruct- 
ing to start execution of time sequence processing 
corresponding to the elapsed time; 

(3) time sequence control information having 
information, which should be played back according 4c 
to the elapsed time, stored in order of playback, the 
time sequence control information including: 

(a) animation control information which describes 
values of the display state information of the 
partial image corresponding to the elapsed time; 45 
and 

(b) time sequence processing control information 
having a name of time sequence information to be 
played back according to the time sequence pro- 
cessing start command. IDs showing contents of 50 
the time sequence information to be played back at 
predetermined times, and playback state informa- 
tion describing how the time sequence informa- 
tion should be played back in accordance with 
display of the partial image shown by the partial 55 
image information in the partial image list; 

(b) a page information read means for reading a piece of 
page information from the disc; 

(c) a display data creation means for creating display data 

of the screen image by placing the partial image shown 60 

by the partial image information on a screen; 
(d; a display means for displaying each display data 

created by the display data creation means; 
(e> an input means for receiving an input signal from an 

operator: and 

<f> a command execution means for detecting the input 
signal inputted by the operator, judging whether the 
input signal corresponds to the command described by 
the control information in the page information read by 
the page information read means, and instructing the 
display data creatioa means to create new display data 
by changing the display state of the corresponding 
partial image according to the display state change 
command when the input signal corresponds to the 
display state change command; the command execu- 
tion means including: 

(1) a first timer for measuring the elapsed time after 
start of activation; 

(2) an animation display command execution unit for 
activating the first timer when an input signal cor- 
responding to the animation display command is 
detected, calculating values of the display state infor- 
mation corresponding to the elapsed time of each of 
the partial images based on the animation control 
information at predetermined times, instructing the 
display data creation means to create new display 
data by changing the display state of the partial 
image according to the calculated values of the 
display state information, and stopping the first timer 
when execution of the animation display command 
is completed; 
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(3) a second timer for measuring the elapsed time after 
the start of activation; and 

(4) a time sequence processing execution unit for 
activating the second timer when an input signal 
corresponding to the time sequence processing start 
command is detected, designating contents of the 
time sequence information corresponding to the 
elapsed time, and stopping the second timer when 
playback of the time sequence information is com- 
pleted; 

wherein the display data creation means creates new 
display data according to the animation display com- 
mand execution unit; 

(g) a page storage means, which is implemented by RAM. 
for storing the page information read by the page 
information read means; 

(h) a time sequence information read means for reading 
the time sequence information designated by the time 
sequence processing execution unit from the time 
sequence information stored as one file in the data 
server via a network, the time sequence information 
having contents in order of playback corresponding to 
the elapsed time and its name being included in the time 
sequence control information; and 

(i) a playback means for playing back the time sequence 
information read by the time sequence information read 
means, and instructing the display means to display the 
time sequence information. 
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[57] ABSTRACT 

A CD-ROM 201 for recording multi-media information of at 
least one page stream. Contents of each page stream are 
successively recorded in the storage areas in the CD-ROM 
201. The following three items are shown by the page 
stream: a partial image list having at least one pair of partial 
image data which describes partial images of one screen 
image and display state information including X-Y 
coordinates. X-Y enlargement rates and a rotational angle 
showing the display state: control information which 
describes command to be executed corresponding to an 
input signal: time line information which describes param- 
eters concerning time sequence processing to be executed 
corresponding to the control information. The display data 
creation unit 213 synthesizes partial images shown by the 
partial image list and creates display data from the partial 
image data and display state information in the page stream 
read from the CD-ROM 201. and writes the display data in 
the display data storage unit 209. 
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njLur, a_, . , Patent and Trademark Office; U.S. DEPARTMENT OF COMMPRCF 

Under the Paperwork Reduction Act of 1SS5, no persons are required to respond to a collection of information unless it displays a valid QMB number. 



REISSUE APPLICATION DECLARATION BY THE INVENTOR 



Docket Number (Optional) 
NAKl-AS70r 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are stated below next to my name. 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first • 

and joint inventor (if plural names are listed below) of the subject matter which is described andciaimed 

in patent number 5,767,845 ^ , granted June '16, 1998 , and for which a 

reissue patent is sought on the invention entitled multx -media information record dev ice and 

A MULTI-MEDIA INFORMATION PLAYBA CK DEVICE ~ ~ ~~ . 

the specification of which " — — _ _ . , 

Jp is attached hereto, 

;£] was filed on as reissue application number / m 

IJ j and was amended on . ~ 

j S (If applicable) 

;5 have reviewed and understand the contents of the above identified specification, including the claims 
";==as amended by any amendment referred to above. ' 
,m 4 acknowledge the duty to disclose information which is materia! to patentability as defined in 
n37CFR1.56. 

□verily believe the original patent to be wholly or partly inoperative or invalid, for the reasons described 
! Below, (Check all boxes that apply.) 

ill I by reason of a defective specification or drawing. 

;rp] by reason of the patentee claiming more or less than he had that right to claim in the patent. 
|x] by reason of other errors. 

At least one error upon which reissue is based is described as follows: 

Claims 1 and 4 were amended by a Rule 312 Amendment on February 2, 1998, to 
j replace "disc" with -recording medium-, but the issued claims did not enter the 
| correction. 

Claims 1, 4, 7, 9, 25, and 25 define page information as "successively" stored in a 
storage area, while the prior art did not require such a description and therefore the 
patentees claimed less than they had a right to claim. 

Claims 19 and 20 define "successively" regarding time sequence information in a 
| storage area, while the prior art did not require such a description and therefore the 
patentees claimed less than they had a right to claim. 



?«l2 en A t!f? ur staten J ent: ™ s form if" estimated to take 0,5 hours to complete. Time will vary depending uoon the needs of the individual 
SI?!;, ASi *3\?^L°l*™%y^ aje/egyired^to compete this form shoujd^e sentto the Chief Information Officer 
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(REISSUE APPLICATION DECLARATION BY THE INVENTOR, page 2) 

All errors corrected in this reissue application arose without any deceptive intention on the part of the 
applicant. As a named inventor, I hereby appoint the following attorney(s) and/or agent(s) to prosecute 
this application and transact aii business in the Patent and Trademark Office connected therewith. 



Docket Number (Optional) 
NAKl-AS70r 



Name(s) 




Registration Number 




Joseph W, 


Price, 25,124 


Michael' J. Moffatt, 


39,304 


Albin H. 


Gess, 25,726 


Bradley D. Blanche,- 


38>-387 


Franklin 


D. Ubell, 27,009 


Gordon ~E. Gray III, 


42,602 



Correspondence Address: Direct all communications about the application to: 
1 I Customer Number 



OR 



Type Customer Number here 



Place Customer Number Bar 
Code Label here 



— i Firm or 
1 Individual Name 



Joseph W. Price, PRICE , GESS & UBELL 



Address 



2100 S.E. Main St., Ste/250 



Address 



City 



Irvine 



State 



CA 



ZIP 



92614 



Country 



USA 



Telephone 



949/261-8433' 



Fax 



949/261-9072 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the 
knowledge that willful false statements and the like so made are punishable by fine and imprisonment, 
or both, under 18 U.S.C. 1001 1 and that such willful false statements may jeopardize the validity of the 
application, any patent issuing thereon, or any patent to which this declaration is directed. 



Fuli name of sole or first inventor (given name, family name) 
Masahiro Oashi 



Inventor's signature 



Residence 
Osaka, Japan 



Date 



Post Office Address 43-22 , Nomuramotomachi , 
Hirakata-shi, Osaka 573-01 Japan 



Citizenship 
Japan 



Full name of second joint inventor (given name, family name) 
Yuki Kusumi 



Inventor's signature 



Date 



Residence 
Kashiba , Japan 



Citizenship 
Japan 

Post Office Address 3-15-4U, Sekiyakita^ asnxx^a-shi, *ara o^-uz o^u 



Fuli name of third joint inventor (given name, family name) 
Masahiro Kawai 



Inventor's signature 



Date 



Residence 
Osaka, Japan 



Citizenship Japan 



Post Office Address ^ 
7-23, Kanou, Himashi-Osaka-shi, Osaka 578 Japan 



Additional joint inventors are named on separately numbered sheets attached hereto. 
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t-uw name of s££S?5? joint inventor (given name, family name) 
Chihiro Kawahara 



Inventor's signature 


Date " . 


Residence 

Osaka, Japan 


Citizenship japan 


Po !l^S|Ady||^ kat ^ otomachi ^ Mor ig Uch i« s i 1 i / osalca 570 Japan 


Full name of tH&$|§int inventor (given name, family name) 
Ryoichi Sasaki 


jwentors signature 


Date 


ijj. Osaka, Japan 


Citizenship Japan. . , 


Pbst Office Address 

U 5-3-207, Nakino-Kitama'chi, Hirakata-shi, Osaka 573 Japan 


Fill! name of sole or first inventor (given name, family name) 


Inventor's signature 


i Residence 


Date 


ifpst Office Address 


.Citizenship 


'Pull name of second joint inventor (given name, family name) 


Inventors signature 


Date' " * . 


Residence 


Citizenship 


Post Office Address 


Full name of third joint inventor (given name, family name) 


inventors signature 


Date 


Residence 


Citizenship - - - 


Post Office Address 
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REISSUE APPLICATION DECLARATION BY THE INVENTOR 



Docket Number (Optional) 
NAKl-AS70r 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are stated below next to my name. 

I believe I am the original, first and sole inventor (if only one name is listed beiow) or an original, first ■ 

and joint inventor (if plural names are listed beiow) of the subject matter which is described and claimed 

in patent number 5,767,845 , granted June 16, 1998 , and for which a 

reissue patent is sought on the invention entitled multi-media information record device and 

A MULTI-MEDIA INFORMATION PLAYBACK DEVICE 

' •- ■ . ,. „— . —■ . - — . „ j 

the specification of which 

HQ is attached hereto. 

D Q was filed on June 2, 2000 as re j ssue application number H 1 581 ' 107 

H and was amended on . 

II (If applicable) 

^ 1 have reviewed and understand the contents of the above identified specification, including the claims, 

□ as amended by any amendment referred to above. 

u i acknowledge the duty to disclose information which is material to patentability as defined in 
37CFR1.56. 

□ 1 verily believe the original patent to be wholly or partly inoperative or invalid, for the reasons described 
f I below. (Check all boxes that apply.) 

S [] by reason of a defective specification or drawing. 

3 H by reason of the patentee claiming more or less than he had the right to ciaim in the patent 
[xj by reason of other errors. 

At least one error upon which reissue is based is described as follows: 

Claims 1 and 4 were amended by a Rule 312 Amendment on February 2, 1998, to 
j replace "disc" with -recording medium--, but the issued claims did not enter the 
| correction. 

Claims 1, 4, 7, 9, 25 , and 25 define page information as "successively" stored in a 
storage area, while the prior art did not require such a description and therefore the 
patentees claimed less than they had a right to claim. 

Claims 19 and 20 define "successively" regarding time sequence information in a 
storage area, while the prior art did not require such a description and therefore the 
patentees claimed less than they had a right to claim. 
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case. Any comments on the amount of time you are required to complete this form should be sent to the Chief Information Officer 
Patent and Trademark Office, Washington, DC 20231 . DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS SEND TO' 
Assistant Commissioner for Patents, Washington, DC 20231 . " 
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(REISSUE APPLICATION DECLARATION BY THE INVENTOR, page 2) 



Docket Number (Optional) 
NAKl-AS70r 



AH errors corrected in this reissue application arose without any deceptive intention on the part of the 
applicant. As a named inventor, I hereby appoint the following attorney(s) and/or agent(s) to prosecute 
this application and transact all business in the Patent and Trademark Office connected therewith. 



Name(s) 


Registration Number 






Joseph w. Price 7 25,124 


Michael J. 


Moffatt, 


39,304 


Albin H. Gess, 25,726 


Bradley D. 


Blanche 


38,387 


Franklin D. Ubell, 27,009 


Gordon : E . 


Gray III, 


42,602 



Correspondence Address: Direct ail communications about the application to: 
□ Customer Number 



OR 



Type Customer Number here 



Place Customer Number Bar 
Code Label here 



□ Firm or 
Individual Name 



Address 



Address 



City 



Country 



Joseph W. Price, PRICE, GESS & UBELL 



21DD S.E. Main St., Ste.*250 



Irvine 



State 



CA 



ZIP 



92614 



USA 



Telephone 



949/261-8433 



Fax 



949/261-9072 



1 hereby declare that ail statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the 
knowledge that willful false statements and the like so made are punishable by fine and imprisonment, 
or both, under 1 8 U.S.C. 1 001 , and that such willful false statements may jeopardize the validity of the 
application, any patent issuing thereon, or any patent to which this declaration is directed. 



Full name of sole or first inventor (given name, family name) 
Masahiro Oashi 



Inventor's signature 



Residence 
Osaka, Japan 



Date 



June 21 , 2000 



Post Office Address 43-22, Nomur aiao tomachi , 
Hirakata-shi, Osaka 573-01 Japan 



Citizenship 
Japan 



Full name of second joint inventor (given name, family name) 
Yuki Kusumi 



Inventor's signature 



Residence 
Kashiha, Japan 




^Cu*Hr>~ * 



Date 



June 21 , 2000 



Citizenship 

Japan 



Post Office Address Sekiyakita f J^snxba-shi, wara 639-02 Japan 



Full name of third joint inventor (given name, family name) 
M&sahiro Kawai 



Inventor's signatun 



Residence 



Hi gashi -Osaka, Japan 



Date 



June 21 , 2000 



Citizenship Japan 



Post Office Address 
7-23, Kanou, Higashi-Osaka-shi, Osaka 578 Japan 



X Additional joint inventors are named on separately numbered sheets attached hereto. 
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run name of s£S6M joint inventor (given name, family name) 

Chihiro Kawahara 



Inventor's signature 



Date 



June 21 , 2000 



Residence 

Osaka , Japan 



Citizenship Jap an 



Po !L»^tekatamotomachi , Moriguchi-shi, Osaka 570 Japan 



Full name of 8$&?j§int inventor (given name, family name) 

, Ryoichi Sasaki 



Jiventor^s signature 



Res! 



Date 



June 21 , 2000 



idence 

Osaka, Japan 



Citizenship Japan . 



J>ost Office Address 

: 5-3-207, Makino-Kitamachi , Hirakata-shi , Osaka 573 Japan 



Jul! name of sole or first inventor (given name, family name) 



Inventor's signature 



3Res!dence 



Date 



4Post Office Address 



Citizenship 



If ull name of second joint inventor (given name, family name) 



Inventor's signature 



Date 



Residence 



Citizenship 



Post Office Address 



Full name of third joint inventor (given name, family name) 



Inventor's signature 



Date 



Residence 



Citizenship 



Post Office Address 



Page 3 of 3 



FORMALITIES LETTER 
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UNITED STATES DEPARTMENT OF COMMERCE 
Patent and Trademark Office 

Address- COMMISSIONER OF PATENT AND TRADEMARKS 
Washington, DC 20231 



APPLICATION NUMBER 



FILING/RECEIPT DATE 



FIRST NAMED APPLICANT 



ATTORNEY DOCKET NUMBER 



09/587,107 



06/02/2000 



Masahiro Oashi 



NAKl-AS70r 



Joseph W Price 
Price Gess & Ubeil 
2100 SE Main Street 
Suite 250 
Irvine, CA 92614 



Date Mailed: 06/21/2000 



□ NOTICE TO FILE MISSING PARTS OF REISSUE APPLICATION 
]U Filing Date Granted 

, ; An application number and filing date have been accorded to this reissue application. The item(s) indicated below 
;™ however, are missing. Applicant is given TWO MONTHS from the date of this Notice within which to file all 

required items and pay any fees required below to avoid abandonment Extensions of time may be obtained by 
M filing a petition accompanied by the extension fee under the provisions of 37 CFR 1136(a). 

□ • The oath or declaration is unsigned. 

^ • To avoid abandonment, a late filing fee or oath or declaration surcharge as set forth in 37 CFR 1 .16(e) of 
^ $130 for a non-small entity, must be submitted with the missing items identified in this letter 

• Assignee's statement under 37 CFR 3.73(b) establishing ownership of the patent is missing. 37 CFR 1 172 
; ~ requires that all assignees consenting to the reissue application establish their ownership interest in the 

y patent by filing in the reissue application a statement in accordance with 37 CFR 3.73(b). 

i J • The balance due by applicant is $ (30 



A copy of this notice MUST be returned with the reply. 




Customer Service Center 
Initial Patent Examination Division (703) 308-1202 

PART 2 - COPY TO BE RETURNED WITH RESPONSE 




1 of 1 



6/20/00 4:44 PM 



NAKl-AS70r 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re Application of: 
Masahiro Oashi et al. 
Serial No.: 09/587,107 
Filed: June 2, 2000 



For: MULTI-MEDIA INFORMATION 

RECORD DEVICE AND MULTI-MEDIA 
INFORMATION PLAY-BACK DEVICE 




Examiner: 

Group Art Unit: 2772 

July 10, 2000 

Irvine, California 92614 



LETTER 

Assistant Commissioner for Patents 
Washington, DC 20231 

Dear Sir: 

In response to the Notice of Missing Parts dated June 21, 2000, enclosed are a 
Declaration and Assignee's Statement Under 37 CFR 3.73(b) for filing in the above-identified 
case together with our check for $130.00 . Please charge any additional fees to our deposit 
account No. 1 6-2462. A copy of this letter is provided for that purpose. 

Also attached are a Consent of Assignee, Claim of Foreign Priority, and Offer to 
Surrender Patent. 

If there are any questions, please do not hesitate to contact the undersigned attorney. 

Very truly yours, 



I hereby certify that this correspondence is being 
deposited with the U.S. Postal Service as first class 
mail in an envelope addressed to Assistant 
Commissioner for Patents, Washington, DC 2023 1 , 
on July 10, 2000, by Daniel Kerby 



PRICE, GESS & UBELL 




Signature 
July 10, 2000 




Date of Signature 



iW, Price, Reg. No. 25,124 
2100 S.E. Main St., Ste. 250 
Irvine, CA 92614 
949/261-8433 




// 
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Patent and Trademark Office, U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. 



REISSUE APPLICATION BY THE ASSIGNEE, 
OFFER TO SURRENDER PATENT 



Docket Number (Optional) 
NAKl-AS70r 



This is part of the application for a reissue patent based on the original patent identified below. 



Name of Patentee(s): 

Masahiro Oashi et al. 



Patent Number 
5,767,845 



Date Patent Issued 
June 16, 1998 



Title of invention multi-media information record device, and a multi-media 

INFORMATION PLAYBACK DEVICE 



Matsushita Electric 
Industrial Co., Ltd. 

1 offer to surrender the original patent. 



. is the assignee of the entire interest in the original patent. 



[3 A certificate under 37 CFR 3.73(b) is attached. 



I am authorized to act on behalf of the assignee. 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under 18 U.S.C. 1001 and that such willful false statements may 
jeopardize the validity of the application, any patent issued thereon, or any patent to which this 
declaration is directed. 



Name of assignee 

MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD. 



Signature of person signing for assignee Date 

I June 21, 2000 

Typed ^j^n^^j^^e and title of person signing for assignee 



Director, Intellectual Property Center 
Authorised Signing Offic e r 



Burden Hour Statement: This form is estimated to take 0.1 hours to complete. Time will vary depending upon the needs of the 
individual case. Any comments on the amount of time you are required to complete this form should be sent to the Chief 
Information Officer, Patent and Trademark Office, Washington, DC 20231. DO NOT SEND FEES OR COMPLETED FORMS 
TO THIS ADDRESS. SEND TO: Assistant Commissioner for Patents, Washington, DC 20231. 



PTQ/SB/96 (2-98) 

Approved for use through 09/30/2000. OMB 0651 -0031 
\o i£/ Patent and Trademark Office; US . DEPARTMENT OF COMMERCE 

\>£\ Under gjj^aperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number 



STATEMENT UNDER 37 CFR 3.73^ 



Applicant: Masahiro Oashi et al. 



Application No, OS/512,302 Filed: August 8, 1995 



Fntitipfi * MULTI-MEDIA INFORMATION RECORD DEVICE, AND A MULTI-MEDIA INFORMATION ... 
Matsushita Electric Industrial Co., Ltd. , a a corporation of Japan 



(Name of Assignee) {Type of Assignee, e.g., corporation, partnership, uni verstiy, government agency, etc.) 

states that it is: 

1. f71 the assignee of the entire right, title, and interest; or 

2. an assignee of an undivided part interest 



in 



U.S. Pat. 5, 767,845 by virtue of either: 



A. i/l An assignment from the inventory } of the patent application identified above. The assignment was recorded in the Patent 

and Trademark Office at Reel 7759 . Frame 0902 , or for which a copy thereof is attached. 

OR 

B. [ ] A chain of titie from the inventor(s), of the patent application identified above, to the current assignee as shown below: 



1. From: To: 



The document was recorded in the Patent and Trademark Office at 

Reel , Frame , or for which a copy thereof is attached. 



2. From: To: 



The document was recorded in the Patent and Trademark Office at 

Reel , Frame , or for which a copy thereof is attached. 



3. From: To: 



The document was recorded in the Patent and Trademark Office at 

Reel , Frame , or for which a copy thereof is attached, 

] Additional documents in the chain of titie are listed on a supplemental sheet. 



[ J Copies of assignments or other documents in the chain of titie are attached. 

The undersigned (whose titie is supplied below) is empowered to sign this statement on behalf of the assignee. 

June 21 , 2000 

Date -^Signature 



Typed or printed name 

Osamu Yaniazaki 

Director, Intellectual Property Center 
Authorized StgiW. Officer 



Burden Hour Statement: This form is estimated to take 0.2 hours to complete. Time-wHLvary_ depending upon the needs of the individual case Any comments on the amount of 
time you are required to complete this form should be sent to the Chief information Officer, Patent and Trademark Office, Washington, DC 20231 DO NOT SEND FEES OR 
COMPLETED FORMS TO THIS ADDRESS. SEND TO: Assistant Commissioner for Patents, Washington, DC 20231. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Application for Reissue of: 
U.S. Patent No. 5,767,845 
Issued: June 16, 1998 
Applicant: Masahiro Oashi et al. 




Serial No.: 08/512,302 

For: MULTI-MEDIA INFORMATION RECORD 
DEVICE, AND A MULTI-MEDIA 
INFORMATION PLAYBACK DEVICE 



CONSENT OF ASSIGNEE 



Assistant Commissioner for Patents 
Washington, DC 20231 

Dear Sir: 

Matsushita Electric Industrial Co. Ltd. of Japan, the Assignee of the entire right, title and 

interest of U.S. Patent No. 5,767,845 by virtue of assignment recorded in the U.S. Patent Office 

hereby consents to the reissue application executed by the inventors of said patent. 

Respectfully submitted 

MATSUSHITA ELECTRIC INDUSTRIAL 
CO., LTD. 



Dated: June 21 



,2000 



By: 



Title: 




Osamu Yamadkf"^ 



Director, Intellectual Property Center 

Authorized Signing Officer 



